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Making  Analysis  Relevant  ' 

Approti^  fesf  piMt&o  ;■  ..(.Xis? 

We’ve  all  seen  this.  Most  experi¬ 
enced  analysts  may  have  experi¬ 
enced  the  following.  The  analy¬ 
sis  team  has  just  finished  briefing  the 
General  on  the  results  of  their  six  month 
study  effort.  The  briefing  went  well 
enough.  It  should  have;  after  all,  the  model 
was  accredited,  the  data  were  vaUdated,  and 
the  peer  review  was  rigorous.  The  General 
had  a  few  polite,  penetrating  questions  fol¬ 
lowed  by,  “Nice  job,”  and  “Keep  up  the 
good  work.”  The  only  actionable  guidance 
from  the  briefing  was  framed  as  a  question, 
“Who  do  you  plan  to  distribute  it  to?”  Deep 
inside,  you  know  now  that  despite  all  the 
hard  work,  this  study  is  not  going  to  change 
the  world.  It  isn’t  going  to  have  a  notice¬ 
able  impact.  There  is  nothing  wrong  with 
it.  It’s  elegant,  but  it  isn’t  “RELEVANT.” 
Tragically,  many  analyses,  particularly  at 
the  top  levels  of  defense  decision  making, 
tend  toward  the  category  of  irrelevancy. 

What  can  make  a  study  irrelevant? 
There  can  be  many  reasons.  Most  obvious 
is  that  it  is  late;  delivering  its  message  after 
the  decisions  are  made.  A  study  can  also  be 
irrelevant  if  it  addresses  the  wrong  ques¬ 
tions  or  if  it  just  doesn’t  address  the  central 
issues.  More  subtly,  a  study  can  be  irrele¬ 
vant  if  in  the  end  game  it  just  lays  there;  not 
offering  an  action  verb  in  its  findings. 

To  be  more  relevant  a  study  should  be 
what  could  be  called  “actionable.”  In 
today’s  Defense  arena,  “actionability”  tends 
to  come  in  two  flavors;  increasing  capabili¬ 
ties  and/or  lowering  costs.  The  Air  Force 
has  coined  a  word  for  making  studies 
“actionable.”  They  call  it  “operationalizing” 
the  study. 

For  example,  another  study  addressing 
the  impact  of  Weapons  of  Mass  Destruction 
on  theater  level  operations  could  tend 
quickly  toward  irrelevancy  unless  it  goes  far 
enough  to  provide  insight  into  what  might 
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derail  the  relevance  of  the  follow  on  study. 
The  old  adage,  “Paralysis  by  Analysis,” 
probably  has  some  roots  in  this  phenome¬ 
non.  Follow  on  studies  that  don’t  help 
move  the  decision  makers  toward  action 
tend  to  become  irrelevant.  The  effort  put 
into  them  becomes  a  substitute  for  progress. 

So,  what  can  an  analyst  do  to  help  make 
a  study  more  relevant? 

Maybe  the  most  useful  approach  is  to 
follow  an  analytic  process,  a  recipe.  Many 
of  the  notable  analysts  who  have  come 
before  us  used  such  recipes.  My  favorite  is 
that  provided  by  John  D.  (Dave) 
Robinson,  Maj  Gen  USA,  Ret.  when  he 
was  the  Director  for  Force  Structure, 
Resources  and  Assessment,  J8.  The  analy¬ 
sis  process  was  to  follow  these  steps  in 
order: 

•  What’s  the  question? 

•  What’s  the  “real”  question? 

•  What  do  the  final  slides  look  like? 

•  What  do  I  already  know? 

•  How  do  I  get  the  remaining  information 
I  need? 


be  done  to  lessen  the  impacts  and  reclaim 
lost  capabilities.  Studying  how  bad  things 
can  get  can  be  eye  opening  and  down  right 
motivational.  But,  unless  the  study  pro¬ 
vides  “actionable”  insights,  unless  it  “opera¬ 
tionalizes”  the  subject,  it  is  just  begging  the 
next  question,  “What  can  be  done  about  it?” 
A  study’s  relevancy  lies  in  its  ability  to 
motivate  action. 

Having  said  that,  I  need  to  acknowledge 
that  quite  often  the  role  for  a  study  is  sim¬ 
ply  to  inform.  Sometimes  static  insight  is 
needed.  Insight  is  a  first  step  to  taking 
effective  action.  But,  the  role  for  such  stud¬ 
ies  is  just  that,  a  first  step.  Too  often,  the 
first  study  may  become  the  format  for  the 
follow-on  study  (“Tell  me  More”).  The 
analyst  needs  to  be  careful  here.  Doing  the 
same  old  trick,  but  with  a  bigger  dog  can 


Let’s  look  at  the  first  two  steps  of  the 
recipe,  ‘What’s  the  Question?”  and 
“What’s  the  Real  Question?” 

Today,  more  than  ever,  the  initial  ques¬ 
tion  asked  of  the  analyst  is  probably  not 
well  focused.  The  “real  questions”  are  elu¬ 
sive.  They  are  elusive  for  a  lot  of  reasons. 
We  don’t  have  a  well  understood  threat. 
We  have  a  host  of  new  missions  to  consid¬ 
er.  Technology  is  perturbing  our  warfight¬ 
ing  processes  faster  that  we  can  characterize 
them.  Basic  military  organizations  and 
institutions  are  in  flux.  Asymmetries 
abound.  The  force  structure  and  the  budget 
is  a  constant  question.  Jointness  seems  to 
pervade  everything,  yet  at  times  it  is  diffi- 
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MORS  1997-98:  “Keeping  MiUtary  Operations  Research  Relevant” 


The  theme  for 
the  1997-1998 
MORS  is 
“Keeping  Military 
Operations  Research 
Relevant.”  A  sub¬ 
theme  that  suggests 
itself  is  “Keeping 
MORS  relevant.” 
There  are  two  major 
ongoing  keys  to  keep¬ 
ing  our  society  rele¬ 
vant.  These  are  peo¬ 
ple  and  vision.  As  for  people,  MORS  must 
have  volunteers  because  MORS  is  an  orga¬ 
nization  of  volunteers.  MORS  represents 
well  over  2500  professionals. 

As  described  in  the  ‘'World  of  MORS,'' 
our  professional  activities  are  managed  by 
a  Board  of  Directors  (BOD)  of  30  opera¬ 
tions  research  professionals  —  28  of  which 
are  elected  to  fill  emerging  vacancies  for  a 
four  year  term  by  the  existing  BOD.  It  is 
essential  that  membership  is  viewed  not  as 
recognition,  but  as  a  responsibility  to  the 
profession  to  work  hard  to  accomplish 
effectively  and  efficiently,  the  purpose  of 
MORS  and  to  keep  MORS  relevant.  The 
BOD  strives  to  insure  that  it  represents  the 
membership  both  in  its  makeup  and  by  its 
actions.  First,  we  strive  to  insure  there  are 
members  from  all  six  Sponsors  —  military 
and  other  government  operations  research 
analysts,  from  inside  and  outside  the 
“Washington  beltway,”  from  military 
offices  in  the  field,  and  from  academia, 
FFRDCs  and  industry  —  all  taking  actions 
together  to  create  and  achieve  a  vision  for 
MORS. 

In  addition  to  the  BOD,  MORS  is  made 
up  of  many  groups  of  volunteers  —  senior 
advisory  groups;  advisory  directors;  non¬ 
board  committee  members;  composite  and 
working  group  chairs,  co-chairs  and  advi¬ 
sors;  Past  Presidents  and  Fellows;  editors 
and  department  editors  of  MORS  publica¬ 
tions;  and  others  whom  respond  to  special 
requests.  To  repeat,  the  key  to  MORS  is 
people  —  volunteers  —  who  work  togeth¬ 
er  to  structure  the  future  vision  of  MORS, 
to  define  a  Plan  of  Action  and  Milestones 
(POA&M)  for  the  near-term  portion  of  the 


vision,  and  to  assist  in  keeping  the  MORS 
vision  relevant. 

The  overarching  boundaries  of  the 
MORS  vision  is  captured  in  the  MORS 
Goals  which  identify  the  purpose  of 
MORS  and  twelve  focus  areas  for  MORS. 
The  MORS  Goals  are  to  enhance  the  quali¬ 
ty  and  usefulness  of  military  operations 
research.  The  Society  endeavors  to: 

•  Understand  and  encourage  responsive¬ 
ness  to  the  needs  of  the  user  of  military 
operations  research. 

•  Provide  opportunities  for  professional 
interchange. 

•  Educate  members  on  new  techniques  and 
approaches  to  analysis. 

•  Provide  peer  critique  of  analyses. 

•  Inform  and  advise  decision  makers  on  the 
potential  use  of  military  operations 
research. 

•  Encourage  conduct  consistent  with  high 
professional  and  ethical  standards. 

•  Recognize  outstanding  contributions  to 
mihtary  operations  research. 

•  Assist  in  the  accession  and  development 
of  career  analysts. 

•  Strive  for  a  membership  which  is  repre¬ 
sentative  of  the  military  operations 
research  community. 

•  Preserve  the  heritage  of  military  opera¬ 
tions  research. 

•  Preserve  the  role  of  MORS  as  a  leader  in 
the  analytical  community. 

•  Encourage  the  use  of  operations  research 
in  support  of  current  military  operations. 

The  process  for  defining,  implementing, 
and  adjusting  the  MORS  vision  for  imple¬ 
menting  our  MORS  Goals  is  also  impor¬ 
tant.  This  process  is  the  MORS  strategic 
planning  process  which  is  provided  over¬ 
sight  by  a  Strategic  Planning  Committee 
that  reports  to  the  President. 

Each  year  for  the  past  several  years,  the 
MORS  BOD  has  conducted  a  strategic 
planning  session  in  June  before  the  annual 
symposium.  The  purpose  of  the  meeting 
was  to  ensure  the  BOD  remained  focused 
on  the  future  as  next  year’s  Executive 
Council,  BOD,  and  committee  members 
were  to  be  determined  in  the  following  few 

3 


days.  As  the  Strategic  Planning  Commit¬ 
tee  chairman.  Dr.  Gerry  McNichols  (twice 
MORS  BOD  member)  pointed  out  a  few 
years  ago,  the  strategic  planning  process  is 
as  important  as  the  strategic  plan.  In  June 
1997,  the  plans  of  actions  and  milestones 
(POA&Ms)  of  the  Executive  Council  and 
their  committees  were  reviewed  to  see  how 
well  MORS  Goals  and  other  objectives 
were  being  achieved.  This  year  the  focus 
was  to  write  POA&Ms  early  and  share 
with  each  Director  so  we  could  work 
together  and  capture  emerging  opportuni¬ 
ties  across  different  committees.  Another 
attempt  to  influence  the  future  direction 
and  vision  of  MORS  was  for  your  Execu¬ 
tive  Vice  President,  Dick  Wiles,  and 
myself  to  brief  “MORS  Directions”  to  not 
only  our  MORS  Sponsors,  but  also  leaders 
and  members  of  FFRDCs  and  analytical 
companies  that  support  MORS  with  their 
members’  sponsored  participation.  The 
feedback  was  useful  and  the  challenges 
tough,  and  the  feehng  emerged  that  MORS 
might  not  always  have  time  to  respond 
efficientiy. 

Based  on  this  feedback  and  lessons 
learned  from  past  Board  strategic  planning 
sessions,  in  December  Ted  Smyth  —  cur¬ 
rent  chair  of  the  Board  Structure  and  Gov¬ 
ernance  Committee  and  past  chair  of  the 
Strategic  Planning  Committee  —  proposed 
that  a  new  volunteer  officer  position  be 

(See  MORS  PRESIDENT,  p,  30) 


Upcoming  MORS  Meetings 

Analysis  Requirements  for  the  Next 
QDR  -  7,  8,  9  April,  The  Johns  Hop¬ 
kins  University  Applied  Physics  Lab, 
Laurel,  MD 

66th  MORS  Symposium  -  23,  24,  25 
June  1998,  Naval  Postgraduate 
School,  Monterey,  CA 

SIMTECH  2007  Session  2  —  Tenta¬ 
tively  scheduled  for  11-13  August 
1998,  Location  to  be  determined 
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1998  —  A  Year  of  Accomplishments  for  MAS 


AS  we  move 
into  1998,  I 
am  reporting 
to  you  that  MAS  is 
healthy  and  all  trends 
are  in  the  right  direc¬ 
tion.  Our  finances  are 
in  good  order,  and  we 
have  a  number  of  new 
initiatives  that  will  be 
realized  during  1998. 
There  are  many  possible  topics  for  this  col¬ 
umn,  but  I  have  selected  two  for  discus¬ 
sion:  The  MAS  National  Conference  and 
Military  Operations  Research  at  non-mili¬ 
tary  universities. 

It  has  been  many  years  since  MAS  has 
hosted  a  conference.  MAS  conferences 
were  quite  common  in  the  1980s,  but  in 
recent  years  we  have  focused  almost 
entirely  on  the  INFORMS  National  Meet¬ 
ings.  In  a  recent  survey,  the  MAS  mem¬ 
bership  indicated  that  a  MAS  conference, 
independent  of  the  INFORMS  National 
Meetings,  was  desirable.  The  idea  for  a 
MAS  National  Conference  was  proposed 
by  my  predecessor  (Steve  Balut),  and  it  is 
being  executed  by  our  Redstone  Arsenal 
Chapter. 

The  Military  Applications  Society  of 
the  Institute  for  Operations  Research  and 
the  Management  Sciences  will  hold  its  first 
National  Meeting  on  19-21  May  1998  on 
the  campus  of  the  University  of  Alabama 
in  Huntsville.  The  theme  of  the  meeting  is 
Going  Forward  Into  The  Future  With  Mili¬ 
tary  Operations  Research.  The  conference 
will  be  hosted  by  the  Redstone  Arsenal  — 
Huntsville  Military  Operations  Research 
Section  (RAHMORS)  of  MAS. 

We  are  soliciting  your  support  for  this 
conference.  The  MAS  Council  is  propos¬ 
ing  an  intense  transfer  of  knowledge  in  an 
academic  setting.  Sessions  and  presenta¬ 
tions  in  the  area  of  contemporary  military 
operations  research  and  management  sci¬ 
ence  are  solicited.  Prospective  participants 
and  attendees  may  contact  either  the  Gen¬ 
eral  Chair,  Anthony  Brinkley,  Teledyne 
Brown  Engineering,  Huntsville,  AL 
[tony_brinkley@pobox.tbe.com],  or  the 
Program  Chair,  Bruce  Fowler,  US  Army 


Aviation  Missile  Command,  ATTN: 
AMSAM-RD-AS,  Redstone  Arsenal,  AL 
35898  [fowler-bw@redstone.army.mil  or 
(205)  876-8173]. 

Our  membership  also  indicated  by  sur¬ 
vey  that  participation  in  a  single 
INFORMS  National  Meeting  per  year  was 
desirable.  In  response  to  this  desire  MAS 
has  not  organized  a  full  program  for  the 
26-29  April  National  Meeting  in  Montreal. 
However,  MAS  has  organized  several 
panel  discussions,  and  the  MAS  Council 
will  be  conducting  business  at  the  meeting. 
Also,  there  will  be  Military  Operations 
Research  sessions  at  this  meeting.  These 
sessions  were  organized  as  part  of  the 
invited  conference  program  by  Kevin  Ng, 
a  long-term  MAS  member  and  distin¬ 
guished  Canadian  colleague.  Information 
about  these  sessions  may  be  obtained 
directly  from  Kevin  [kevinng@dms- 
hp.ora.dnd.ca].  I  look  forward  to  seeing 
you  in  Montreal,  and  I  invite  all  current 
MAS  member  and  interested  colleagues  to 
our  business  meeting  and  reception,  which 
will  be  held  on  the  Monday  evening  of  the 
conference. 

My  second  topic  for  this  column  is  Mil¬ 
itary  Operations  Research  in  non-military 
universities.  I  use  the  term  non-military 
because  all  of  our  service  academies  and 
military  graduate  schools  have  integrated 
Military  Operations  Research  throughout 
their  curricula.  These  schools  continue  to 
produce  the  practitioners  and  leaders  of  our 
profession.  There  are  “pockets”  of  Mili¬ 
tary  Operations  Research  at  other  universi¬ 
ties,  but  I  am  not  aware  of  any  new  pro¬ 
grams  that  have  appeared  in  recent  years.  I 
am  happy  to  report  that  a  new  program  is 
imminent. 

As  a  direct  outgrowth  of  a  MORS  Edu¬ 
cational  Colloquium,  George  Mason  Uni¬ 
versity  in  Fairfax,  Virginia,  has  designed 
and  is  implementing  a  new  curriculum  in 
Military  Operations  Research.  The  Mas¬ 
ters-level  curriculum  will  be  implemented 
in  the  Department  of  Operations  Research 
and  Engineering  within  the  School  of 
Information  Technology  and  Engineering. 
The  degree  offers  a  number  of  elective 
options,  but  the  foundation  is  centered 


around  two  core  courses:  MOR  Effective¬ 
ness  Analysis  and  MOR  Cost  Analysis. 
The  curriculum  is  currently  being  reviewed 
by  senior  members  of  MAS  and  MORS, 
and  the  first  students  will  be  admitted  in 
the  Fall  of  1998.  Details  of  the  program 
may  be  obtained  from  Prof.  Karla  Hoff¬ 
man  [khoffman@vmsl.gmu.edu],  the 
Chair  of  the  Department  of  Operations 
Research  and  Engineering. 

I  run  the  risk  of  being  accused  of  adver¬ 
tising  for  George  Mason  University,  but 
that  is  not  the  point  of  my  column.  The 
point  is  that  our  efforts  in  trying  to  influ¬ 
ence  educational  institutions  to  meet  the 
needs  of  our  profession  are  achieving 
results.  The  efforts  initiated  by  Yupo 
Chan  and  Kenneth  Konwin  kindled  the 
idea  and  had  an  impact  on  the  George 
Mason  University  decision  to  develop  its 
Military  Operations  Research  curriculum. 
I  commend  Yupo  and  Kenneth  for  all  of 
their  hard  work  on  the  MORS  Education 
Committee.  The  students  are  our  future, 
and  a  continued  focus  on  the  primary  input 
(i.e.,  students)  will  assure  quality  output 
(practitioners  and  leaders). 

I  am  interested  in  reporting  other  devel¬ 
opments  relating  to  Military  Operations 
Research  education.  I  welcome  your  input 
on  these  issues.  As  usual,  feel  free  to  con¬ 
tact  me  by  Internet  [gulledge@gmu.edu]. 
The  MAS  Council  continually  monitors 
the  MAS  Listserver,  and  you  can  always 
contact  us  or  the  membership  by  this 
means.  If  you  are  not  already  a  member  of 
the  list,  you  can  subscribe  by  sending  the 
following  message  to  majordomo@ 
mat.gsia.cmu.edu: 

subscribe  mas  Your  Name,  Title 
<  youmame@domain.org> 

If  you  have  problems,  send  a  note  to 
Philipp  Djang.  He  doubles  as  the  modera¬ 
tor  of  the  list.  I  will  use  the  listserver  to 
provide  information  of  interest  to  the  Mili¬ 
tary  Operations  Research  community  as  it 
is  passed  to  me.  I  encourage  you  to  do  the 
same.  O 


Tom  Gulledge 

MAS  President 
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VEEPS  PEEP 


Preparing  the  Military  Operations  Research  Society 
for  the  21st  Century 


TO  paraphrase 
the  theme  for 
the  66th  Mili¬ 
tary  Operations 
Research  Society 
Symposium,  as  we 
approach  the  year 
2000,  it  is  also  a  good 
time  to  take  stock  as 
to  where  the  future  of 
the  Society  lies.  Cer¬ 
tainly,  the  catalyst  for  any  robust  organiza¬ 
tion  is  an  active,  vibrant,  involved  mem¬ 
bership,  which  accepts  the  organization’s 
vision  and  actively  supports  the  goals  and 
objectives  to  achieve  that  vision.  In 
MORS  we  have  such  a  membership.  It’s 
not  just  a  few  pulling  their  weight,  but  the 
active  involvement  of  so  many  which  has 
been,  and  will  continue  to  be,  the  key  to  the 
survival  and  continued  growth  of  our  orga¬ 
nization. 

The  challenge  for  MORS  leadership  — 
as  we  turn  the  century  —  is  to  keep  that 
momentum  going.  We  have  built  a  reputa¬ 
tion  as  a  world  class  organization  which 
quickly  reacts  and  provides  quality  prod¬ 
ucts  to  our  Sponsors’  needs.  Our  annual 
Symposium  has  covered  the  full  spectrum 
of  analysis  thanks  to  the  hardworking  and 
dedicated  efforts  of  the  members  who  have 
served  in  the  capacity  of  CompositeAVork- 
ing  Group  (CGAVG)  leaders.  Even  more 
important,  however,  is  the  vital  contribu¬ 
tion  of  all  those  who  have  supported  the 
Society  in  making  their  presentations  at  the 
MORS  Symposium,  for  this  is  the  entree 
—  the  food  that  provides  the  nourishment 
for  our  Society. 

Our  meeting  operations  do  not  suffer 
from  a  lack  of  topics  for  Mini-Symposia 
and  Workshops;  in  fact,  the  challenge  fac¬ 
ing  us  on  the  Board  is  limiting  the  number 
of  meetings  within  our  Charter  in  the  near 
future.  Our  publications,  the  PHALANX 
and  the  MOR  journal,  continue  to  be  filled 
with  quality,  relevant  articles.  We  always 
have  an  abundance  of  well  deserving 
Board  of  Director  candidates  who  have 


surfaced  from  the  membership  mass.  And 
most  recently,  the  membership  committee 
under  the  leadership  of  LCDR  Katie 
Thurman  assembled  an  outstanding  slate 
of  nominations,  from  which  the  Board 
selected  our  newest  MORS  Fellows:  DR 
Jacqueline  Henningsen,  FS,  Chris  Fos- 
sett,  FS,  and  Vern  Bettencourt,  FS. 
While  congratulations  are  in  order  for  the 
new  inductees  into  that  august,  revered 
elite  —  a  hearty  well  done  is  due  to  the 
membership  committee  who  did  such  an 
outstanding  job  in  identifying  a  candidate 
list  with  such  high  qualifications  and  cre¬ 
dentials. 

What  are  the  nutrients  that  will  keep  our 
organization  strong?  How  can  we  keep  the 
membership  base  rejuvenated  across  the 
many  analytical  fronts,  so  that  the  best  ana¬ 
lytical  efforts  continue  to  be  brought  to  the 
forefront  and  highlighted?  Perhaps  those 
answers  can  be  found  in  the  new  sciences. 
We  recently  have  focused  on  the  applica¬ 
tion  of  the  new  sciences  to  analysis,  or  vice 
versa,  but  what  if  we  applied  some  of  those 
concepts  to  our  organization  itself? 

Let’s  then  examine  some  of  the  life  sus¬ 
taining  ingredients,  which  will  continue  to 
cause  the  Society  to  grow.  Specifically,  if 
we  treat  our  organization  as  a  natural 
chaotic  process  that  continues  to  reshape 
and  revitalize  itself,  three  key  organization¬ 
al  areas  emerge  —  namely  information, 
relationships,  and  vision. 

Information  —  the  Lifeblood 

Information  has  been  characterized  in 
the  new  sciences  as  the  ‘lifeblood’  of  an 
organization.  It  is  the  energy  source, 
which  in  fact  enables  the  organization  to 
live  and  produce.  It  is  what  generates 
activity  within  the  organization,  the  life 
resource  that  allows  the  processes  of 
renewal  and  revitalization  to  occur.  In  our 
Society,  it  may  be  the  sharing  of  a  relevant 
piece  of  analysis,  the  publication  or  presen¬ 
tation  of  analytic  techniques,  the  formula¬ 
tion  of  analytical  approaches,  or  the  shar¬ 
ing  and  exchange  of  ideas  at  Symposia  and 


Workshops.  Information  the  P/MLAA^, 
the  MOR  journal,  the  presentations  at 
MORSS,  the  proceedings  of  our  work¬ 
shops,  and  the  various  literature  published 
by  our  membership. 

Relationships  —  the  Distribution 
Foundation 

Relationships  are  not  the  building 
blocks  of  our  organization,  but  are  more 
like  a  circulatory  system.  They  are  the 
manner  in  which  information  is  distributed 
and  shared  within  our  Society,  the  key  con¬ 
duits  across  all  segments  within  and  exter¬ 
nal  to  our  organization.  They  are  the  Com- 
positeAVorking  Groups,  the  subgroups  of 
workshops,  the  Senior  Advisory  Groups, 
the  Prize  paper  process,  the  committees  of 
our  Board,  as  well  as  our  website  and  e- 
mail  distribution  system. 

It  is  also  how  we  interact  externally  to 
other  organizations  and  societies,  such  as 
MAS  and  INFORMS.  At  the  summer 
1997  Board  Meetings,  we  formed  a  new 
committee  to  examine  our  relationships 
and  how  we  liaison  with  other  organiza¬ 
tions.  The  committee  has  recently  been 
examining  policy  and  guidance  on  how  to 
enter  business  agreements  with  such  orga¬ 
nizations. 

We  have  made  significant  strides  in  the 
area  of  ‘virtual’  relationships  the  past  cou¬ 
ple  of  years  —  with  the  establishment  of  a 
website  that  at  first  was  not  much  more 
than  a  ceremonial  electronic  glossy  adver¬ 
tisement.  Our  electronic  media  committee, 
currently  led  by  Glen  Johnson,  and  the 
MORS  staff  has  continued  to  expand  the 
utility  of  the  web  to  the  point  that  it  has 
become  a  vital  and  accepted  tool.  Current 
information  is  displayed  pertaining  to 
forthcoming  MORS  events  —  the  invita¬ 
tion  letter,  terms  of  reference  (TOR),  appli¬ 
cation  forms,  and  meeting  Read-Aheads  — 
proceedings  of  completed  workshops,  and 
our  Monthly  Activity  Reports  (MAR).  It 
will  no  doubt  continue  to  grow  and  react  to 
change,  to  provide  the  means  to  get  timely, 
(See  VEEPS  PEEP,  p.30) 


CAPT  Lee  Dick 
Secretary 
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Some  Comments  on  the  State  of  Operations  Analysis 

Senior  Lecturer  Wayne  P.  Hughes,  Jr.,  FS 
Naval  Postgraduate  School 


Professor 

Wayne  P. 
Hughes 


An  extraordi¬ 
nary  meta¬ 
morphosis  is 
underway  in  Opera¬ 
tions  Analysis  (OA) 
that  reaches  well  out¬ 
side  our  profession 
and  is  wrapped  up  in 
questions  about  its 
proper  boundary. 
Higher  education 
everywhere  is  threat¬ 
ened  with  a  massive 


transformation,  as  old  disciplines  such  as 
physics,  chemistry,  mathematics,  biology, 
and  what  could  be  called  classical  engi¬ 
neering  are  confronted  by  new  subject  mat¬ 
ter  that  is  in  great  demand  but  is  so  inter¬ 
disciplinary  that  the  traditional  academic 
departments  have  difficulty  organizing  and 
teaching  it,  for  instance  computer  technolo¬ 
gy,  information  technology,  systems  engi¬ 
neering  and  command  and  control.  New 
subject  matter  threatens  the  old  order 
because  it  is  seen  outside  of  academia  to  be 


where  the  future  lies. 

What  we  think  of  now  as  interdiscipli¬ 
nary  studies  may  become  new  or  transcen¬ 
dent  academic  disciplines  or  professions.^ 
At  least  one.  Command  and  Control  (C^) 
(to  include  communications),  is  probably 
now  sufficiently  structured  to  acquire  the 
characteristics  of  an  academic  discipline. 
C^  is  a  burgeoning  technical  subject 
extending  outside  the  boundaries  of  leader¬ 
ship  and  management.  Its  military  roots 
are  rapidly  spreading  into  the  world  of 
business  and  commerce.  More  difficult  to 


assess  is  that  hodge-podge  called  computer 
science,  which  as  a  practical  field  of 
endeavor  seems  to  spawn  offspring  even 
while  it  remains  marginally  accepted  as  a 
respectable  academic  disciphne.  These  are 
treacherous  waters.  Even  though  I  hold 
OA  up  as  an  exemplar  of  a  successful  new 
academic  discipline  that  first  arose  after 
World  War  II  at  the  Naval  Postgraduate 
School  and  Case  Institute,  there  are  tradi¬ 
tionalists  in  academia  who  still  gag  at 
expressing  OA  and  discipline  in  the  same 
breath.  Many  universities  deliver  the  core 
material  of  OA  under  other  rubrics  such  as 


industrial  engineering  and  management 


science,  while  the  US  Naval  Academy 
teaches  OA  in  its  Department  of  Mathe¬ 
matics. 

Indeed,  it  is  hard  to  know  the  bound¬ 
aries  of  contemporary  Operations  Analysis. 
For  instance,  does  OA  include  Information 
Technology  (IT),  or  is  IT  a  new  field  that 
embraces  many  of  OA’s  techniques?  Mod¬ 
em  OA  leans  so  heavily  on  FT  and  comput¬ 
er  science  that  differentiating  among  them 
is  contentious  in  the  academic  world. 
Almost  certainly  the  impact  and  influence 
of  IT,  broadly  construed,  already  exceeds 
my  OA  profession’s  effects.  The  reason 
for  this  is  that  every  activity  (i.e.,  thought 
process  and  operation)  starts  with  an  infor¬ 
mation  input.  Every  activity.  One  might 
suppose  that  whoever  controls  the  informa¬ 
tion  controls  the  world,  except  that  the 
usual  goal  of  IT  has  been  for  no  one  to 
control  and  everyone  to  share. 

The  problem  can  be  traced  to  the  con¬ 
cept  of  ‘‘model.”  Operations  Analysis 
invented  the  term,  even  though  dynamic 
representations  of  phenomena  such  as 
Newton’  laws  of  motion  existed  long 
before  they  were  called  models  of  reality. 
Now  models  are  embraced  by  nearly 
everyone,  and  modeling  is  done  in  many 
places  for  many  purposes  that  are  unques¬ 
tionably  outside  of  OA,  CADCAM  and 
computer  artwork,  for  instance.  Too  many 
analysts  have  come  explicitly  or  implicitly 
to  regard  Operations  Analysis  as  the  art 
and  science  of  building  realistic  models 
that  aspire  to  be  more  and  more  representa¬ 
tive  of  the  phenomena  they  describe,  with 
only  passing  regard  for  the  utility  and  effi¬ 
ciency  of  their  representations  in  helping 
decision  makers.  This  is  a  big  problem.  In 
my  view,  OA  must  surrender  proprietor¬ 
ship  of  modeling  as  our  thing.  OA’s 
uniqueness  and  value  he  elsewhere. 

So  what  can  we  say  about  the  nature  of 
OA  and  its  contribution  to  society?  My  vis¬ 
its  to  Singapore’s  military  and  educational 
institutions  has  given  me  some  contrasting 
viewpoints  and  helped  to  clarify  where  we 
in  the  military  OA  profession  are  headed, 
and  tentatively,  how  to  describe  our  future 
directions  and  contributions. 

At  the  most  general  level,  the  purpose 
of  all  OA  is  to  help  people  think  and  act 


more  wisely.  But  this  general  aim  also 
applies  to  all  education,  to  doctrine,  and  to 
plans  and  operations  of  all  descriptions. 
The  central  purpose  of  OA  is  to  help  deci¬ 
sion  makers  make  better  immediate  choic¬ 
es  of  plans  and  actions.  Of  course  OA  now 
extends  its  influence  well  outside  military 
operations,  but  that  is  not  where  my  practi¬ 
cal  or  academic  experience  lies.  I  have 
pondered  the  question  as  it  pertains  to  mili¬ 
tary  planning  and  operations,  so  that  is 
what  I  will  discuss,  while  adding  that  these 
interpretations  are  probably  generalizable 
to  commercial  affairs.  Regarding  the  role 
of  OA  models,  Military  Modeling  for 
Decision  Making^^^  says: 

“In  operations  research,  the  goodness 
of  a  model  is  judged  by  how  well  it 
achieves  its  purpose.  This  is  no  casu¬ 
ally  selected  criterion.  Operations 
research,  whether  military  or  other¬ 
wise,  is  an  applied  science  more  akin 
to  engineering  and  economics  than  to 
fluid  mechanics  or  psychology.  In  the 
physical  sciences  a  model  is  usually 
judged  on  the  basis  of,  first,  how  truly 
it  portrays  the  phenomenon  it  repre¬ 
sents,  and  second,  its  range  of  apphca- 
bility.  Operations  research  exists  to 
support  better  decisions.  This  is  not 
the  same  thing  as  a  search  for  truth, 
which  is  a  good  bit  harder.  Many 
times  our  standard  of  goodness  is  dis- 
satisfyingly  vague  —  who  would  not 
prefer  an  image  of  universality  such 
as,  say,  the  laws  of  motion?  In  our 
field  of  operations  research  we  will 
have  unkind  things  to  say  about  “uni¬ 
versal”  models  which  try  to  serve  too 
many  purposes.” 

The  essence  of  Operations  Analysis  is 
associated  with  decision  making,  not 
model  building.  Thus,  we  can  exclude  two 
very  large  arenas  supported  by  modeling: 

1,  Education  and  training  devices,  sys¬ 
tems,  and  activities,  including  training 
simulators,  instructional  systems  devel¬ 
opment,  and  instrumented  ranges  used 
for  training. 

2.  Information  technology  systems  and 
activities,  embracing  information  war- 
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fare  (or  conflict),  information  opera¬ 
tions,  and  the  Communications  of  Com¬ 
mand  and  Control  (C^).  IT  deals  with 
the  acquisition,  transfer,  fusion,  display, 
and  dissemination  of  information. 

Properly  construed,  IT  is  decision  neu¬ 
tral,  for  when  it  incorporates  a  decision 
support  system  it  invades  the  domain  of 
OA.  When  IT  specialists  dabble  in  deci¬ 
sion  making  algorithms  they  are  likely  to 
bring  trouble,  because  all  except  the  most 
routine  and  mindless  decision  making 
involves  skills  outside  the  purview  of  most 
experts  in  IT  system  design. 

Part  of  the  problem  stems  from  shared 
use  and  ambiguity.  Instrumented  ranges 
are  employed  for  controlled  experiments  as 
weU  as  training.  War  games  are  used  both 
to  test  new  tactics  and  teach  them.  Infor¬ 
mation  display  systems  can  usefully  incor¬ 
porate  a  decision  support  system,  especial¬ 
ly  when  such  a  system  is  a  mere  network 
of  remotely  located  displays  for  decision 
makers,  as  opposed  to  a  decision  aid  which 
helps  to  analyze  the  consequences  of  a 
future  decision.  In  Singapore,  some  new 
computer  based  systems  that  gather, 
process,  and  display  information  also 
incorporate  decision  algorithms.  At  the 
very  least,  the  information  is  displayed  for 
operators  who  make  their  own  decisions 
based  on  the  information  presented.  The 
power  of  the  system  is  achieved  when  the 
decision  is  instantly  relayed  to  other  deci¬ 
sion  makers  at  other  terminals  tied  into  the 
network,  and  when  aggregated  results  are 
displayed  up  the  line.  Anyone  who  books 
airline  reservations  will  immediately  rec¬ 
ognize  this  as  a  description  of  his  or  her 
network.  Indeed,  the  blend  of  information 
technology  with  widely  decentralized  deci¬ 
sion-making  now  dominates  transport 
operations  of  every  description. 

What,  then  is  the  domain  of  military 
OA?  What  kinds  of  decisions  does  it  sup¬ 
port?  One  at  least  ought  to  summarize  them 
to  indicate  military  OA’s  sweeping  applica¬ 
tions,  even  after  excluding  training  and 
information  technology  from  its  domain.  A 
robust  categorization  is  found  in  Military 
Modeling  for  Decision  MakingX^^  In  order 
to  distinguish  the  differing  characteristics  of 
the  analysis  involved,  the  monograph  lists 
these  applications: 

•  Battle  Planning.  Preparation  for  wartime 
operations,  based  on  friendly  and  enemy 
orders  of  battle,  and  the  existing  strategic 
or  tactical  environment; 


•  Wartime  Operations.  The  conduct  of 
war.  Distinguished  from  battle  planning 
by  available,  current,  wartime  data  and 
known,  immediate  military  objectives; 

•  Weapon  Procurement.  The  selection 
from  among  competing  weapon  systems 
or  characteristics  for  procurement  deci¬ 
sions; 

•  Force  Sizing.  The  decision  of  how  many 
weapons  systems  of  which  types  to  (1) 
operate,  (2)  support,  and  (3)  procure  in 
the  future,  either  in  the  defense  establish¬ 
ment  as  a  whole  or  in  a  major  compo¬ 
nent  such  as  the  army  or  the  nuclear 
weapons  arsenal; 

•  Human  Resource  Planning.  The  design 
and  operation  of  manpower,  personnel, 
training,  and  assignment  systems; 

•  Logistics  Planning.  The  design  and  oper¬ 

ation  of  all  manner  of  military  logistic 
support;  and 

•  National  Policy  Analysis.  Supra  military 
actions  that  influence  or  are  influenced 
by  military  considerations  such  as  arms 
treaties  or  subsidies  of  commercial  trans¬ 
portation. 

All  of  these  applications,  but  most  obvi¬ 
ously  human  resource  and  logistics  plan¬ 
ning,  depend  on  prompt  information  acqui¬ 
sition  and  display,  and  therefore  are 
enhanced  by  information  technology. 
Nevertheless  it  is  well  to  keep  a  clear  dis¬ 
tinction  between  IT  and  OA.  When  an  IT 
system  displays  sufficient  information  for  a 
decision  at  a  terminal,  then  an  operator  can 
take  an  appropriate  action.  When  the  deci¬ 
sion  is  immediately  transmitted  to  every 
operator  in  a  network  and  when  the  infor¬ 
mation  is  aggregated  in  useful  ways  for 
high  level  managers,  then  the  system  is 
valuable  almost  beyond  exaggeration,  yet 
without  any  OA  whatsoever  having  been 
performed.  I  have  mentioned  booking  air¬ 
line  reservations  to  show  what  IT  has 
achieved  for  scheduling.  The  same  value 
of  information  technology  applies  to  the 
transfer  of  funds,  stocks,  bonds,  and  futures 
in  what  with  the  aid  of  IT  has  become  a 
worldwide  network  of  financial  informa¬ 
tion  exchange.  Aggregations  of  informa¬ 
tion,  such  as  the  Dow  Jones  Averages  and 
current  values  of  mutual  funds  are  comput¬ 
ed  and  displayed  almost  continuously. 

These  IT  functions  do  not  make  OA- 
type  decisions.  OA  enters  when  airline 
bookings  lead  to  a  carefully  analyzed 
change  of  flight  schedules  and  prices  in  the 
face  of  competition  and  likely  future  cus¬ 


tomer  demand.  OA  occurs  when  it  helps 
an  investor  makes  his  decision  based  on 
not  merely  the  latest  stock  quotation  but 
also  on  an  appraisal  of  economic  condi¬ 
tions  and  a  personal  investment  policy.  It 
is  here  that  the  IT  expert  in  computer  pro¬ 
gramming  would  do  well  to  pause  before 
he  builds  a  decision-making  algorithm  into 
his  system.  The  algorithm  is  in  the  domain 
of  OA,  and  if  not  done  with  considerable 
skill  it  will  be  counterproductive.  The 
interface  is  doubtless  a  fuzzy  one,  which 
apparently  is  why  Singapore  labels  the 
development  of  information  systems  with 
decision  making  features  “systems  engi¬ 
neering.”  I  think  this  is  a  felicitous  term  for 
the  function,  but  I  wish  you  to  recognize 
that  systems  engineering  blends  together 
two  disciplines  or  skills,  IT  and  OA. 

I  hope  my  profession  (and  academic 
discipline  if  such  it  is)  will  see  the  impossi¬ 
bility  of  claiming  information  technology 
as  its  own.  Our  reward  is  from  aiding  bet¬ 
ter  decisions,  most  especially  those  that 
exceed  routine  operations  but  which  are 
sensitive,  immediate,  often  urgent,  and  of  a 
unique  and  singular  nature.  Understanding 
the  role  of  OA  wiU  dampen  our  ardor  for 
bigger  and  more  comprehensive  models, 
for  the  core  value  of  OA  does  not  lie  in 
model  building  but  in  helping  with  better 
decisions.  The  best  way  to  think  of  an 
operations  analyst  is  as  an  alter  ego  to  the 
decision  maker  who  serves  him  with  disci¬ 
plined,  objective,  time-consuming  thought 
that  the  decision  maker  could  do  for  him¬ 
self  if  only  he  had  the  time.  We  are  also 
somewhat  like  Jiminy  Cricket  on  the 
shoulder  of  Pinocchio  serving  as  his  con¬ 
science,  at  least  to  the  extent  of  promoting 
objectivity  as  the  best  long-term  policy. 
Information  technology  cannot  make  that 
claim,  for  its  special  role  is  as  the  eyes  and 
ears,  and  sometimes  the  voice,  of  the  deci¬ 
sion  maker.  Neither  education  nor  doctrine 
can  make  that  claim,  for  they  are  not  on 
Pinocchio ’s  shoulder,  but  like  Gepetto  are 
removed  in  time  and  space  from  the  scene 
of  action,  and  must  impart  more  general 
wisdom. 

But  the  important  thing  and  the  hard 
part  is  not  in  setting  up  domains  in  which 
training,  doctrine,  information  technology, 
and  Operations  Analysis  play  separate 
roles.  That  is  the  weakness  of  contempo¬ 
rary  academic  departments  at  most  univer¬ 
sities.  The  hard  part  is  to  solve  highly 

(See  COMMENTS,  p.  20) 
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MOR  FORUM 


Gotterdammerung  for  Military  OR  Education 


The  MOR 

Forum  was 
conceived  in 
irreverence,  as  a  sort 
of  op-ed  page  where 
scholars  and  practi¬ 
tioners  of  Military 
Operations  Research 
could  offer  candid 
opinions  and  judg¬ 
ments  on  the  profes¬ 
sional  issues  of  the 
day.  The  aim  implied, 
of  course,  is  to  pro¬ 
voke  thoughtful  and 
reflective  dialogue,  and  thus  promote  the 
intellectual  health  and  growth  of  the  disci¬ 
pline.  The  piece  appearing  here,  by  Dr. 
Dave  Bitters  of  USACGSC,  goes  to  the 
core  of  this  objective,  by  examining  the 
nexus  between  MOR  and  the  larger  body  of 
military  leadership  it  serves.  To  what  extent 
does  the  contemporary  military  leader  need 
exposure  to  analytical  tools  and  techniques 
to: 


David  L.  Bitters 

US  Army 
Command  and 
General  Staff 
College 


•  Conduct  systematic  military  planning  and 
operations? 

•  Attack  problems  in  a  structured,  methodi¬ 
cal  way? 

•  Interpret  critically  the  results  of  rigorous 
analysis  imbedded  in  complex  processes 
like  resource  allocation,  force  planning, 
weapons  modernization,  and  the  like? 


Grave  questions,  to  be  sure,  and  the 
trends  evidenced  in  Bitters’  account  may 
alarm  some.  He  raises  an  important  collat¬ 
eral  question,  too,  about  the  projected  avail¬ 
ability  of  focused  military  education  for 
analysts  themselves  on  complex  military 
applications.  This  is  the  other  side  of  the 
same  coin  denominating  how  tight  the  seam 
between  military  operations  and  Military 
Operations  Research.  The  issue,  in  short,  is 
relevance.  Keep  those  cards  and  letters 
coming. 

—  Dom  Crawford 


Franz  Kafka’s  surreal  tale  The  Meta¬ 
morphosis  begins  with  Gregor 
Samsa,  the  protagonist,  awakening 
one  morning  after  a  bad  dream  to  find  him¬ 


self  turned  into  a  large  beetle.  Lying  on  his 
hardshelled  back  with  his  small  legs  point¬ 
ing  upward,  he’s  unable  to  turn  over  or  lift 
himself  off  his  bed.  Worse,  he’s  locked  the 
door  to  his  room,  so  none  of  his  family  can 
get  to  him.  He  frets  that  in  this  unfortunate 
state  he’ll  be  unable  to  dress  himself  in  time 
to  catch  the  7:00  train  to  work.  This  will 
result  in  all  sorts  of  unpleasantness.  Indeed 
he’s  soon  visited  by  his  firm’s  head  clerk, 
who  demands  a  reason  for  his  tardiness.  He 
pleads  that  he  views  punctuality  as  a  solemn 
obligation  but  begs  to  be  excused,  just  this 
once,  due  to  circumstances  beyond  his  con¬ 
trol.  He  explains  that  it  won’t  happen  again, 
that  he’s  sure  he  can  get  himself  together  in 
time  to  catch  the  8:00  train  —  and  please 
give  his  respects  to  the  Chief.  Of  course, 
the  clerk  hears  nothing  but  gibberish. 

I  thought  of  this  stoiy  when  my  employ¬ 
er  offered  me  the  opportunity  to  change 
occupations. 

A  little  background  would  be  in  order. 
From  1981  until  the  summer  of  1996  I  was 
a  full-time  instructor  of  operations 
research/management  science-related  cours¬ 
es  at  the  US  Army  Command  and  General 
Staff  College.  The  College’s  continuing 
interest  in  my  services  flowed  primarily 
from  my  involvement  with  a  course  called 
Resource  Planning  and  Allocation.  This 
was  an  amalgam  of  basic  OR/MS  concepts 
(decision  analysis,  probability  and  statistics, 
linear  regression,  economic  analysis.  Pro¬ 
gram  Evaluation  Review  Technique/Critical 
Path  Method  (PERT/CPM)),  and  some 
material  on  force  development  and  financial 
management.  During  a  decade  and  a  half  I 
taught  the  OR/MS  portion  to  more  than 
3500  officers. 

Though  it  was  a  bitter  pill  for  some  to 
swallow,  most  students  gave  this  course 
favorable  reviews.  While  30  percent  or  so 
admitted  to  being  “mathematically  chal¬ 
lenged,”  most  seemed  to  enjoy  the  rigor  and 
the  prospect  that  if  they  followed  the  rules 
they  could  get  the  right  answers.  A  typical 
end-of-course  comment  read  something  like 
this:  “I  worked  my  tail  off.  But  it  was 
worth  it  and  I  learned  something.”  Of 
course  there  always  were  some  who  assert¬ 
ed,  “I’ll  never  use  this  stuff.” 


I  also  taught  several  specialty  courses  to 
smaller,  self-selecting  audiences.  These 
included  Topics  in  Operational  Decision 
making  and  a  two-term  sequence  called 
Military  Operations  Research.  The  Col¬ 
lege  calls  itself  the  senior  school  for  tactics 
in  the  Amy,  so  I  made  a  conscious  effort  to 
align  the  content  of  these  courses  with  this 
focus. 

Most  US  Army  officers  lack  the  mathe¬ 
matical  background  that  their  counterparts 
in  the  former  Soviet  Union  apparently  had. 
Moreover,  those  who  do  have  mathemati¬ 
cal  training  often  forget  much  of  it  during 
their  field  tours.  Few  know  anything  about 
specialized  topics  such  as  Z-transforms  for 
probability  modeling.  So  the  first  part  of 
Military  Operations  Research  amounted  to 
a  calculus  and  probability  refresher.  It 
looked  like  an  engineering  mathematics 
course,  without  apparent  relevance  to  mili¬ 
tary  problems.  Without  it,  though,  the 
models  in  the  second  course  would  have 
been  inaccessible  to  most  students.  This 
made  it  necessary  to  omit  a  number  of  oth¬ 
erwise  interesting  and  relevant  modeling 
topics. 

The  contents  of  the  second  part  of  Mili¬ 
tary  Operations  Research  included  disper¬ 
sion,  vulnerability  and  lethality  models, 
multiple  and  area  target  models,  sortie 
models,  stochastic  duel  models  and  force- 
on-force  attrition  models. 

These  models  tell  a  story.  They  give 
insights  into  how  to  operate  the  system,  and 
expose  problems  and  issues.  While  the 
validity  of  some  of  them  in  the  modern 
context  is  a  matter  of  on-going  discussion, 
they  provide  a  source  of  hypotheses  to  test 
and  a  framework  for  structuring  thinking 
about  tactical  problems. 

Enrollment  in  the  specialized  courses 
was  fairly  constant  until  about  four  years 
ago,  when  the  College  began  to  mandate 
certain  branch-specific  “electives.”  Courses 
not  on  the  short  list  (mine  included)  experi¬ 
enced  noticeable  drops  in  enrollment. 

Early  in  1996  the  contents  of  Resource 
Planning  and  Allocation  came  under  seri¬ 
ous  review.  Some  had  expressed  concern 
that  certain  topics  of  general  interest 
weren’t  receiving  enough  instructional 
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emphasis.  As  the  Command  and  General 
Staff  Officers’  Course  (CGSOC)  common 
curriculum  at  best  involves  a  zero-sum 
game  in  terms  of  instructional  hours,  the 
quantitative  methods  hours  became  a  covet¬ 
ed  prize. 

Shortly  before  College  management 
moved  to  drop  these,  a  Training  and  Doc¬ 
trine  Command  (TRADOC)  senior  officer 
briefed  the  faculty  on  certain  aspects  of  a 
new  concept  called  Army  XXI.  With  the 
enthusiasm  of  one  used  to  directing  (but  not 
necessarily  doing)  big  projects,  he  offered 
that  “winning  the  information  war”  would 
be  fraught  with  new  challenges,  as  the  Army 
ventured  into  the  complexities  of  traffic 
management  in  communications  networks. 

I  listened  in  vain  during  this  talk  for  evi¬ 
dence  of  even  rudimentary  understanding  of 
queuing  theory  and/or  network  modeling, 
technologies  that  have  been  around  for  at 
least  thirty  years.  I  also  found  myself 
scratching  my  head  in  bewilderment;  con¬ 
currently  the  College  was  debating  cutting 
the  instruction  on  network  modeling,  among 
other  things. 

Ft.  Leavenworth  hosted  the  MORSS 
conference  in  1996.  I  looked  forward  to 
participating  in  it  fully.  However,  shortly 
before  the  end  of  the  academic  year  I 
learned  that  Resource  Planning  and  Alloca¬ 
tion  was  dead.  So  during  MORSS  week  I 
found  myself  attending  a  training  course  on 
force  management.  This  was  to  prepare  me 
to  teach  the  replacement  course:  Resource 
Planning  and  Force  Management.  I 
approached  this  viewgraph-rich  experience 
with  vigor,  if  not  with  complete  comprehen¬ 
sion.  Meanwhile,  I  sandwiched  attendance 
at  MORSS  sessions  into  my  schedule  when¬ 
ever  possible. 

Several  weeks  later  management 
informed  me  the  College  had  abolished  my 
teaching  job.  No  need  for  an  operations 
research  instructor  when  there’s  no  opera¬ 
tions  research  instruction. 

It  was  at  this  point  that  I  thought  of  Gre¬ 
gor  Samsa. 

You  might  ask,  Who  cares?  Lots  of  peo¬ 
ple  have  been  “downsized.”  The  Cold 
War’s  over;  the  government’s  going 
through  a  contraction;  operations 
researchers  (and  teachers  of  operations 
research)  are  as  dispensable  as  anyone  else. 
Life’s  tough.  We  cope  and  move  on. 

Still,  while  recognizing  the  dangers  of 
generalizing  from  a  small  sample  size,  I 
submit  that  there’s  a  cautionary  tale  buried 
in  my  experience. 


(1)  Since  the  Vietnam  era,  operations 
research  methodology  has  been  embed¬ 
ded  in  the  Army  culture.  Every  Com¬ 
mand  and  General  Staff  College 
(CGSC)  graduate  (whether  of  the  resi¬ 
dent  course  or  one  of  the  correspon¬ 
dence  programs)  got  approximately 
twenty  contact  hours  (or  equivalent)  of 
instruction  in  quantitative  methods. 
You  could  expect  mid-grade  Army  offi¬ 
cers  working  for  or  with  you  to  have  at 
least  a  rudimentary  technical  under¬ 
standing  of  the  phrase  to  optimize. 
They  would  appreciate  that  often  the 
most  challenging  decision  problems 
involve  multiple  criteria.  They  would 
understand  what  it  means  to  talk  about 
a  good  decision  with  a  bad  outcome. 
They  would  know  that  estimation  based 
on  averages  might  lead  to  underestima¬ 
tion  for  planning  purposes.  At  one  time 
in  their  lives  they  would  have  demon¬ 
strated  proficiency  in  constructing  and 
solving  a  PERT/CPM  network.  Some¬ 
time  after  the  millennium  senior  com¬ 
manders  will  find  that  people  on  their 
staffs  no  longer  know  these  things. 

(2)  The  operations  research  methods  we 
taught  to  the  mid-level  leadership 
involved  no  mathematical  skills  beyond 
simple  algebra.  Bright  high-school  stu¬ 
dents  could  have  mastered  the  mechan¬ 
ics.  The  value  of  the  OR  methodology 
is  its  rigorous,  structured  approach  to 
problem  formulation  and  solution.  As 
most  people  who  have  done  “real”  OR 
recognize,  you’re  often  80  percent  of 
the  way  to  the  solution  of  a  problem, 
once  you’ve  defined  it  sufficiently  to 
formulate  a  model.  By  dropping  the 
quantitative  methods  instruction  from 
the  CGSC  curriculum,  we  can  no 
longer  be  sure  that  our  mid-grade  offi¬ 
cers  have  the  “thinking  skills”  this  dis¬ 
cipline  provided. 

(3)  My  college  ROTC  program  thirty-five 
years  ago  had  the  title  Military  Science 
and  Tactics.  Today,  I  hear  almost  no 
one  at  CGSC  speak  of  military  science. 
However,  I  often  hear  phrases  such  as 
military  art  and  operational  art.  Has  the 
Army  abandoned  the  notion  that  the 
rigors  of  the  scientific  method  apply  to 
its  business?  Is  it  a  waste  of  time  to 
teach  the  systems  approach  to  operators 
of  the  system?  If  so,  organizations  such 
as  MORS  would  do  well  to  rethink 
their  charters.  If,  on  the  other  hand, 
there  is  a  military  science,  it  isn’t  perco¬ 


lating  into  the  schoolhouse  very  well. 

(4)  Likewise,  people  in  the  tactics  and  lead¬ 
ership  departments  speak  of  the  art  of 
command.  Is  this  something  real  that 
can  be  communicated  and  for  which  one 
can  be  apprenticed?  This  remains  to  be 
seen.  Last  summer  I  did  extensive  sta¬ 
tistical  analysis  on  a  data  set  derived 
from  battles  conducted  at  the  National 
Training  Center.  Recognizing  the  ambi¬ 
guities  hidden  in  the  raw  data,  I  wanted 
to  see  if  any  variant  of  the  Lanchester 
models  proposed  by  anyone  during  the 
past  forty  years  provided  a  reasonable 
fit.  With  the  opposing  force  (OPFOR), 
red  force,  data  I  did  find  a  pattern.  With 
the  blue  force  data  the  casualties 
appeared  to  bear  no  discernible  rela¬ 
tionship  to  the  starting  size  of  the 
OPFOR,  the  blue  force,  or  any  combina¬ 
tion  or  multiple  thereof.  How  do  we 
explain  this?  Is  it  because  our  comman¬ 
ders  are  inadequately  prepared  to  do 
their  job?  Is  it  because  they  aU  do  it  dif¬ 
ferently,  with  unpredictable  skill?  I 
don’t  know,  but  my  study  raises  interest¬ 
ing  questions  about  the  art  of  command. 

(5)  There  exists  a  body  of  scholarly  litera¬ 
ture  under  the  rubric  of  Military  Opera¬ 
tions  research.  Here  I  have  in  mind  the 
various  combat  models.  Few  institu¬ 
tions  of  higher  learning  offer  instruction 
in  this  technology,  and  their  numbers  are 
shrinking  (the  Air  Force  Institute  of 
Technology,  for  instance,  will  soon 
close  its  doors).  While  I  spend  most  of 
my  time  in  academic  administration 
now,  I  did  offer  the  Military  Operations 
Research  sequence  at  CGSC  last  spring. 
Only  one  student  enrolled  in  Military 
Operations  Research  If  the  models 
course.  So,  while  this  sequence  isn’t 
dead  yet,  it’s  clearly  comatose.  My 
guess  is  that,  like  General  MacArthur’s 
Old  Soldier,  it  will  just  fade  away.  This 
seems  strange,  since  the  concepts  that 
form  its  contents  are  embedded  in  many 
of  the  models  and  simulations  the  Army 
depends  on  in  the  development,  procure¬ 
ment  and  decision  cycles  (Kafka’s  tale 
comes  to  mind  again). 

People  at  higher  levels  than  I  who  are 
paid  more  than  I  make  the  tough  choices 
concerning  budget  allocation  and  course 
content.  I  won’t  try  to  second-guess  them. 
But  I  do  think  the  MORS  community  should 
be  aware  of  what  they’ve  done,  and  begin 
preparing  for  the  likely  consequences.  O 
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Quick  Response  Simulation:  Laughlin  AFB  Capacity  Analysis 


Dr.  James  K.  Lowe 

United  States  Air 
Force  Academy 


Capt  Michael  R, 
Weeks 

United  States  Air 
Force  Academy 


Introduction 


Current  software  accessibility  dra¬ 
matically  reduces  the  time  and 
effort  required  to  develop,  validate, 
and  visualize  models  supporting  manageri¬ 
al  decisions.  Much  of  the  simulation  litera¬ 
ture  focuses  on  spreadsheet  applications 
for  quick-turn  probabilistic  analyses;  how¬ 
ever,  with  modem  tools  and  techniques, 
discrete-event  simulation  software  can  now 
be  used  for  projects  that  must  be  completed 
in  minimum  time. 

The  authors  recently  were  involved  in 
such  a  short-term  project.  The  leadership 
at  Laughlin  Air  Force  Base,  an  undergrad¬ 
uate  pilot  training  base  in  Del  Rio,  Texas, 
saw  the  writing  on  the  wall  regarding  dra¬ 
matic  increases  in  pilot  production  and 
wanted  a  quick  analysis  of  their  airfield’s 
capacity.  Specifically,  the  Laughlin  leader¬ 
ship  was  not  sure  that  the  47^^  Flying 
Training  Wing  could  meet  the  training 
requirements  projected  by  headquarters  for 
the  next  five  years.  The  Air  Force  needed 
more  pilots,  and  they  needed  them  in  a 
hurry.  Plans  were  being  formulated  to 
increase  output  for  Undergraduate  Pilot 
Training  (UPT).  While  increased  student 
load  would  normally  be  welcomed,  for  the 
remaining  three  UPT  bases  (dwindled  from 
five  in  1983)  the  increased  demand 
appeared  to  exceed  their  physical  capacity. 
Unfortunately,  the  deterministic  spread¬ 
sheet  planning  model  employed  to  deter¬ 
mine  maximum  enrollments  did  not  cap¬ 
ture  the  important  limiting  effect  of 
variation.  The  authors  were  sent  to  Laugh¬ 
lin  AFB  to  develop  a  more  realistic  model 
of  the  maximum  sortie  capacity  at  Laugh¬ 
lin. 


Background 


The  United  States  Air  Force  trains  hun¬ 
dreds  of  pilots  every  year  and  the  manage¬ 
ment  of  this  force  is  one  of  the  greatest  per¬ 
sonnel  problems  faced  by  the  Air  Force 
leadership.  Market  forces  continually 
change  the  demand  for  pilots  in  the  airline 
industry.  During  hiring  booms,  the  major 
airlines  will  hire  over  300  pilots  per  month, 
most  of  them  military.  Additionally,  the 
unpredictable  government  budget  cycle 
means  that  predicting  the  need  for  pilots  is 
notoriously  difficult.  In  times  of  national 
crisis  or  a  pro-defense  administration,  the 
need  is  great.  In  times  of  stability  and 
peace,  the  defense  budget  cuts  dig  deepest 
into  education  and  training  programs. 

Military  downsizing  efforts  after  the 
end  of  the  Cold  War  reduced  the  Air  Force 
pilot  production  to  approximately  500 
pilots  per  year  in  1994  and  1995  (see  Fig¬ 
ure  1).  In  addition  to  the  drawdown,  the 
airline  industry  was  in  a  recession  and  hir¬ 
ing  was  nil  at  the  major  airlines.  As  a 
result,  most  pilots  (70-80%)  were  accept¬ 
ing  a  pilot  bonus  and  agreeing  to  stay  in  the 
Air  Force  through  14  years  of  service  at  the 
end  of  their  initial  training  commitment 
(see  Figure  2). 

During  this  period  the  Air  Force 
enjoyed  excess  capacity  at  its  pilot  training 
bases,  which  prompted  the  Base  Realign¬ 
ment  and  Closure  Commission  to  list  UPT 


bases  at  the  top  of  their  closure  lists. 
Williams  AFB  had  already  closed,  then 
Reese  AFB  was  added  to  the  closure  list  in 
1993.  Thus,  40%  of  the  UPT  capacity  was 
eliminated  in  rapid  succession  (Sheppard 
AFB,  a  joint  NATO-US  training  base,  also 
remained  open). 

Shortly  after  Reese  was  added  to  the 
closure  list,  the  Air  Force  began  to  realize  a 
problematic  trend;  pilots  were  leaving  the 
Air  Force  at  an  alarming  rate.  The  major 
airlines  had  begun  hiring  in  large  numbers, 
and  the  production  pipeline  had  been  pro¬ 
ducing  a  fraction  of  the  pilots  needed.  The 
pilot  bonus  acceptance  rate  plunged  from 
over  70%  in  1994  to  approximately  30%  in 
1997.  Given  these  conditions,  the  Air 
Force  planned  to  increase  the  UPT  produc¬ 
tion  to  1100  pilots  per  year  by  1999,  more 
than  twice  the  number  produced  in  1994 
and  1995. 

Previous  Studies 

Could  the  remaining  three  UPT  bases 
sustain  1 100  graduates?  Keep  in  mind  that 
the  current  15%  student  attrition  rate 
requires  that  T-37  schedulers  plan  for  near¬ 
ly  1300  pilot  candidates.  Headquarters’ 
planners  felt  that  this  production  level  was 
feasible.  Analysts  projected  that  a  produc¬ 
tion  rate  of  1 100  pilots  per  year  equated  to 


Figure  1 

Source;  Air  Force  Personnel  Center 
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Pilot  Bonus  Acesptanee  Rato  - 1989-1997 


Y«ar 


Figure  2 

Source:  Air  Force  Personnel  Center 


92%  of  overall  capacity.  Laughlin  plan¬ 
ners  wondered  whether  the  maximum 
capacity  estimates  were  realistic.  This 
level  of  performance  would  be  the  highest 
operations  tempo  for  UPT  bases  since  the 
Vietnam  era,  even  higher  than  the  Reagan 
build-up  of  the  mid-80s.  In  addition,  the 
maximum  capacity  estimates  were  derived 
from  simulations  run  during  the  1970’s. 
At  that  time,  only  two  aircraft  (T-37  and 
T-38)  were  involved  in  training  pilots. 
Many  factors  had  changed  in  20  years, 
such  as  environmentally-friendly  airspace 
restrictions  and  the  implementation  of  the 
Specialized  Undergraduate  Pilot  Training 
(SUPT)  program.  The  Air  Force  now 
relies  upon  three  aircraft  (T-37,  T-38  and 
T-1)  to  train  its  pilots.  The  addition  of  the 
T-IA  Jayhawk  trainer  added  a  new 
dynamic  to  the  air  operations  while  creat¬ 
ing  additional  infrastructure  requirements. 
Further,  during  this  production  build-up 
the  Air  Force  must  transition  into  an 
entirely  new  trainer  platform  as  the  Joint 
Primary  Aircraft  Trainer  System  (JPATS) 
replaces  the  T-37. 

The  existing  training  capacity  model 
was  a  spreadsheet  program  based  upon 
many  variables,  including  weather  attri¬ 
tion,  maintenance  attrition,  and  miscella¬ 
neous  attrition  for  such  factors  as  pilot  ill¬ 
ness  and  scheduling  difficulties.  The 
historical  data  for  these  factors  was  thor¬ 
ough  and  stable.  However,  one  important 
assumption  of  the  model  specifies  that  air¬ 
craft  takeoff  every  2.5  minutes.  Thus,  the 
maximum  capacity  was  based  upon  an 
assumption  that  had  not  been  tested  opera¬ 
tionally  within  the  current  SUPT  training 
syllabus.  Obviously  sustained  capacity 
was  highly  sensitive  to  this  number.  For 
example,  during  a  twelve-hour  day,  a  take¬ 
off  every  2.5  minutes  would  generate  288 
training  sorties,  while  a  takeoff  every  3 
minutes  would  generate  240  sorties,  a  dif¬ 
ference  of  approximately  20%. 

Laughlin’s  Predicament 

Initially,  Air  Education  Training  Com¬ 
mand  (AETC)  projected  a  gradual  increase 
in  pilot  production  to  about  900  pilots 
annually  by  2001 .  Although  this  would  be 
a  large  increase,  the  Laughlin  leadership 
felt  that  this  production  level  was  manage¬ 
able.  However  in  late  1996,  airhne  hiring 
increased  and  the  pilot  bonus  acceptance 
rate  plummeted  to  less  than  30%.  The  Air 
Force  would  have  to  replace  even  more 
pilots  than  previously  planned.  AETC 


raised  the  planned  production  numbers  to 
1100  pilots  annually  by  1999.  Laughlin’s 
leadership  felt  that  this  number  might  not 
be  feasible  under  the  current  methods  of 
operation.  However  to  verify  their  intu¬ 
ition,  they  desired  an  objective  analysis  of 
the  maximum  capacity  given  airspace,  area 
locations,  runway  configurations  and  pat¬ 
tern  restrictions.  The  Laughlin  leadership 
was  not  interested  in  sortie  scheduling, 
maintenance  schedules,  aircraft  or  crew 
availability,  attrition  rates,  weather  factors 
(other  than  daylight  hours),  or  any  limiting 
factors  that  could  be  resolved  with  addi¬ 
tional  funds.  The  focus  was  to  determine 
the  absolute  upper  limit  given  perfect 
external  conditions  and  little  change  to  the 
operational  structure. 

Realizing  that  this  project  required 
external  expertise,  a  former  faculty  mem¬ 
ber  from  the  Air  Force  Academy  called  the 
USAFA  Department  of  Management  for 
help.  The  Department  constructed  a  con¬ 
sulting  team  that  consisted  of  a  team  leader 
with  problem  framing  and  modeling  exper¬ 
tise  from  the  Management  department,  a 
Computer  Science  faculty  member  with 
modeling  and  simulation  expertise,  a  Man¬ 
agement  instructor  with  subject  matter 
expertise  (recent  AETC  T-38  instructor 
pilot),  and  a  recent  Operations  Research 
graduate.  The  team  was  established  quick¬ 
ly  and  began  work  to  resolve  this  complex 
issue  almost  immediately. 

As  initially  presented  to  the  consulting 


team,  the  problem  was  “simple”;  calculate 
the  capacity  of  the  base  and  have  a  report 
ready  within  30  days.  Our  team  quickly 
realized  a  problem  that  our  USAFA  stu¬ 
dents  often  complain  about:  time.  We  did 
not  have  time  to  provide  a  full  solution 
within  the  timeframe  requested.  Fortu¬ 
nately,  Laughlin  personnel  were  gracious 
and  agreed  to  explain  the  problem  and 
await  our  assessment. 

Narrowing  the  Scope 

The  team’s  first  priority  was  to  under¬ 
stand  daily  operations.  This  required  a 
quick  trip  to  Laughlin  AFB.  Since  Ran¬ 
dolph  AFB  in  San  Antonio  (home  of 
Headquarters,  AETC)  is  on  the  way  to 
Laughlin,  a  side  trip  to  Randolph  AFB  to 
visit  with  AETC’s  resident  sortie  rate 
expert  was  in  order.  The  team  walked  the 
flightline,  sat  in  the  air  traffic  control 
tower,  talked  with  schedulers,  and  wit¬ 
nessed  students  from  briefing  to  debrief¬ 
ing.  These  on-site  observations  helped 
frame  the  problem  and  visualize  some  of 
the  key  issues.  The  team  quickly  realized 
the  magnitude  of  this  difficult  task.  In 
order  to  meet  the  time  restrictions  for  this 
project,  it  was  necessary  to  limit  the  scope 
of  the  analysis.  Laughlin  leadership 
agreed  that  a  complete  model  of  limited 
scope  was  more  valuable  than  an  incom¬ 
plete  attempt  at  a  comprehensive  model. 

{See  QUICK  RESPONSE,/?.  31) 
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A  Cybernetic  Explanation  of  the  Paradigm  Shift  in  the 
Praxis  of  Complex  Dynamics  in  the  Combat  Simulation 
Environment:  The  Case  of  Joint  Warfare  System  (JWARS) 


Introduction 

Operations  re¬ 
search  (OR), 
development 
of  combat  Models  and 
Simulations  (M&S) 
and  methods  of  ana¬ 
lyzing  the  complexity 
of  the  simulated  bat¬ 
tlefield  have  under¬ 
gone  a  paradigm  shift 
over  the  last  30  years.  Currently,  the 
M&S  community  is  embarking  on  a  new 
paradigm  of  M&S  analytical  development 
called  JWARS.  It  is  felt  that  JWARS  will 
be  the  first  of  a  new  paradigm  of  analytic 
models  that  can  better  address  the  complex 
warfight  issues  of  the  OR  and  M&S  com¬ 
munity. 

The  objective  of  this  cybernetic  explo¬ 
ration  of  the  environment  of  complexity 
within  the  combat  simulation,  and  the 
exploration  of  the  complexity  analysis 
within  the  Military  OR  community  is  to 
outline  a  strategy  for  understanding  the 
paradigm  shift  that  has  occurred.  This  shift, 
as  evidenced  by  symposia  like  MORSS, 
indicate  a  new  paradigm  in  the  praxis  of 
working  in  the  OR  environment.  The 
implications  for  the  future  of  military  OR 
will  be  told  in  the  applications  of  the  new 
class  of  simulations  such  as  JWARS. 

The  Paradigm  Shift  of  OR 

Since  World  War  II,  the  paradigm  of 
OR  has  focused  on  optimization,  and  as 
computer  technology  became  more  of  a 
common  tool,  the  development  of  combat 
models  and  simulations  followed  this  para¬ 
digm.  Limitations  of  computer  technology 
and  the  growing  use  of  OR  tools  con¬ 
tributed  to  better  understanding  of  various 
optimization  tools  and  techniques. 

The  OR  method  used  the  reductionist 
method  when  analyzing  weapon  systems, 
tactics  and  doctrine.  And,  the  charm  for  us 
was  always  success  for  the  systems  ana¬ 
lyst,  computer  programmer,  and  the  deci¬ 
sion  makers  who  had  a  measure  of  effec¬ 
tiveness  to  help  them  choose  which 
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alternative  weapon  system  to  purchase. 

But,  consider  the  Military  OR  commu¬ 
nity  as  a  closed  system.  When  it  started  it 
had  variety.  The  OR  community  was  com¬ 
posed  of  different  scientists,  civilian  and 
military  thinkers,  historians,  academics  and 
lay  persons.  It  started  as  disjoint  incremen¬ 
talism.  This  variety  provided  a  success 
factor  in  the  early  days  of  military  OR  that 
could  be  attributed  to  problem  solving 
skills  from  that  wide  variety  of  mixed-dis- 
ciplined  teams,  as  well  as  a  sound  under¬ 
standing  of  the  practical  issues  to  be 
addressed  obtained  from  user  experience. 

Over  the  last  30  years  that  variety  has 
slowly  decreased  due  to  focused  training  in 
the  “science”  of  OR.  This  decreased  vari¬ 
ety  was  seen  by  some  as  limiting  the  ability 
of  the  OR  community  to  conduct  complex 
warfare  analysis. 

Since  the  1960’s  the  discipline  of  Mili¬ 
tary  OR,  assisted  by  the  growth  of  comput¬ 
er  technology,  developed  a  foundation  or 
legacy  of  combat  models  and  simulations 
that  allowed  the  operations  researcher  to 
better  understand  the  limitations  within  the 
realm  of  modeling  real  world  events.  The 
number  of  models  and  simulations  grew 
from  a  handful  to  hundreds. 

In  the  1980’ s  the  Department  of 
Defense  embarked  on  a  bold  approach  to 
bring  Artificial  Intelligence  (AI)  applica¬ 
tions  into  decision  making,  OR,  data  man¬ 
agement,  combat  service  support  and  com¬ 
bat  simulations.  Add  to  this  AI  push  (and 
credit  is  given  to  the  insights  of  GEN  Max 
Thurman),  the  creation  of  the  Army 
Model  Improvement  Program,  the  collapse 
of  the  Soviet  Union  and  the  end  of  the 
Cold  War,  and  the  entire  concept  of  Mili¬ 
tary  OR  was  beginning  to  be  reexamined 
from  within  and  without.  However,  the 
paradigm  of  OR  was  not  easy  to  change. 

A  concept  was  that  AI  would  bring  the 
simulations  closer  to  this  battle  truth  by 
addressing  that  troublesome  error  term. 
That  push  into  AI  did  not  succeed  in 
replacing  the  error  term.  It  did  provide  us 
with  the  metaphors  that  brought  the  Mili¬ 
tary  OR  community  to  where  it  is  today. 
And,  while  many  may  see  that  AI  push  as  a 


failure  to  deliver  certain  products,  perhaps 
the  key  legacy  was  the  new  language  and 
metaphors  that  we  use  today  that  allow  us 
to  have  a  more  open,  less  reductionist, 
approach  to  problem  solving  —  using 
combat  simulations. 

In  1992,  a  warning  was  issued  from  the 
Deputy  Under  Secretary  of  the  Army  for 
Operations  Research,  that  the  military  OR 
community  could  go  the  way  of  the 
dinosaurs  if  we  were  not  careful.  This 
warning  seemed  to  hearken  to  the  early 
days  of  OR  with  multi-disciplined  teams  of 
experts  solving  complex  problems.  And,  it 
was  during  this  time  frame  that  the 
Defense  Modeling  and  Simulation  Office 
(DMSO)  formed  and  not  long  afterwards 
that  the  Army’s  Model  and  Simulation 
management  began  critical  actions  that 
have  shifted  the  OR  paradigm  back  toward 
the  multi-disciplined  concept. 

The  DMSO  gathered  experts  from 
around  DoD  to  attack  how  to  think  about 
and  how  to  develop  the  next  generation  of 
problem  solving  Models  and  Simulations. 
The  Army  Model  and  Simulation  Office 
(AMSO)  established  a  group  of  Standards 
Categories  and  Standards  Category  Coor¬ 
dinators  composed  of  a  mix  of  experts 
from  all  functional  areas  of  combat  Model¬ 
ing  and  Simulation  research  and  develop¬ 
ment  to  attack  in  order  to  set  the  stage  for 
the  next  generation  of  tools  and  techniques 
to  use  in  OR  and  in  building  Models  and 
Simulations. 

The  impact  of  DMSO,  AMSO  and  sim¬ 
ilar  activities  can  be  seen  if  you  think  of 
the  Military  OR  community  as  a  machine 
with  input,  just  like  the  Models  and  Simu¬ 
lations  it  produces.  This  community  —  the 
black  box  of  OR  —  is  composed  of  many 
groups  of  humans,  who  together  describe 
different  transformations  of  transforma¬ 
tions  with  their  behavior  over  the  nature  of 
OR.  And,  when  you  examine  this  OR 
black  box  over  the  past  30  years  or  even 
the  past  50  years,  we  can  see  that  the 
insides  of  the  black  box  of  professional 
military  and  civilian  operations  researcher 
is  autopoietic.  That  is,  the  insides  of  the 
black  box  are  self-creating  the  boundaries 
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or  walls  of  that  black  box  through  their 
dynamics  of  operation.  A  key  part  of  this 
autopoietic  change  is  the  inclusion  of  the 
groups  such  as  DMSO  and  AMSO.  It 
appears  that  the  appropriate  requisite  vari¬ 
ety  has  shifted. 

The  Paradigm  Shift  of  M&S 

Today’s  OR  problems  are  just  as  com¬ 
plex  as  they  were  in  World  War  n.  Often 
the  problem  statement  is  ill-structured, 
composed  of  the  perceived  situation,  the 
desired  situation,  and  the  actual  situation, 
as  well  as  involving  international  elements, 
having  conflicting  value  systems,  multiple 
policy  actors,  and  exhibiting  self-reference. 
The  combination  of  the  above  is  the  diffi¬ 
culty,  the  complexity,  and  how  the  OR 
black  box  poses  the  complex  analysis 
question  to  be  answered  is  today’s  chal¬ 
lenge.  A  companion  of  this  challenge  is  to 
understand  the  requisite  variety  needed 
within  our  combat  models  and  simulations. 
The  answer  to  this  next  challenge  of  requi¬ 
site  variety,  is  in  the  very  large  combat 
simulations  like  JWARS. 

JWARS  not  only  is  a  new  combat 
model  or  simulation,  but  a  metaphor  for  a 
new  paradigm  shift  in  thinking  about  and 
building  such  systems.  In  the  last  30  years 
of  computer  combat  Models  and  Simula¬ 
tions  the  OR  paradigm  focused  on  attrition- 
based  Models  and  Simulations,  and  provid¬ 
ed  a  rich  literature  to  examine  the  limits  of 
algorithmic  truth  approaching  battle  truth. 
JWARS  has  shown  an  evolution  of  that 
paradigm  with  its  information-based  sys¬ 
tem  that  is  more  like  the  elements  of  real 
life.  That  is  one  key  element  of  the  para¬ 
digm  shift  in  M&S. 

The  foundation  of  the  complex  legacy 
Models  and  Simulations  could  also  be 
described  as  non-trivial  machines.  That  is, 
the  history  of  the  processes  inside  the 
model  or  simulation  was  not  observed  or 
transparent  to  an  outside  observer  or  the 
results.  One  built  these  machines  from 
deterministic  parts,  sequentially  linking 
processes,  which  are  now  being  replaced  in 
new  language  and  in  implementation.  This 
other  key  element  of  the  M&S  paradigm 
shift  is  the  OR  communities’  acceptance  of 
object  oriented  programming  (another  lan¬ 
guage  legacy  from  the  early  efforts  in  AJ 
programming). 

Along  with  improving  computer  tech¬ 
nology,  the  increased  variety  of  the  OR 
community,  and  object  oriented  thinking 


and  practices,  we  have  a  paradigm  shift  in 
M&S.  But,  not  without  a  price. 

The  workings  of  a  non-trivial  machine 
are  difficult  if  not  impossible  to  explain 
and  understand.  The  human  and  human 
organizations  are  non-trivial  machines. 
Now,  with  systems  such  as  JWARS  we  are 
creating  more  real  life  expressions  of 
human  decision-making,  of  command  and 
control.  So,  a  third  key  to  the  new  para¬ 
digm  of  M&S  is  for  JWARS  to  be  able  to 
show  causality  for  its  decision  making 
processes.  This  means  creating  a  complex 
machine  that  does  not  exhibit  the  limita¬ 
tions  of  non-trivial  machines.  JWARS 
must  be  transparent  in  its  complexity,  yet 
not  have  complexity  drive  events  into 
chaotic  conditions.  This  is  a  process  of 
managing  complexity,  similar  to  managing 
a  complex  chemical  reaction.  JWARS 
then,  is  evolving  the  requisite  variety  to 
contain  a  potentially  highly  unstable  reac¬ 
tion.  It  is  putting  constraints  on  chaos. 

The  shift  in  OR  and  M&S  paradigm 
together  have  launched  a  new  era  of  capa¬ 
bility  driven  concepts  and  research  pro¬ 
grams  that  have  buffeted  the  threat  of  OR 
going  the  way  of  the  dinosaurs.  Thus,  we 
have  the  first  critical  paradigm  shift  that  is 
necessary  in  the  next  evolutionary  step 
leading  the  M&S  community  into  the  next 
century. 

The  Paradigm  Shift  in  Analysis 

The  OR  community  has  the  responsibil¬ 
ity  of  advising  decision  makers  on  the 
degree  of  belief  to  ascribe  to  the  results  of 
its  M&S  tools.  We  ask  such  questions  as: 
How  do  we  measure  the  results  of  a  com¬ 
bat  model  or  simulation?  What  is  the  dif¬ 
ference  between  the  empirical  result!  of 
testing  military  systems  and  the  synthetic 
testing  inside  a  model  or  simulation? 

During  the  Cold  War  era,  analysis  had  a 
decided  bias  toward  weapon  systems 
acquisition.  A  debate  ran  through  the  OR 
community  about  whether  or  not  to  use  a 
simple  model  or  a  more  complex  model. 
Accuracy  in  data  quality  was  a  routine 
plague.  Eighth  grade  arithmetic  leveled 
the  language  barrier  between  analyst  and 
decision  maker.  The  scientific  method  was 
not  used,  relying  on  more  art  than  science. 
Analysis  was  not  geared  toward  problem 
solving,  but  more  hke  a  detective’s  investi¬ 
gation  of  facts.  Analysis  was  not  looking 
for  tradeoffs  fi-om  branch  to  cross-branch 
to  campaign  levels.  Models  as  large  as 


JWARS  often  saw  the  model  builder  more 
interested  in  the  model  than  in  the  analysis. 

Perhaps  during  that  time  the  often  heat¬ 
ed  discussions  and  debates  about  what  OR 
techniques  were  being  used  and  what 
M&S  results  could  be  believed,  probably 
contributed  the  most  value  to  the  improv¬ 
ing  analysis  products.  Thanks  in  part  to 
efforts  from  DMSO  and  AMSO,  and  to  the 
debates  and  discussions  from  MORS  con¬ 
tributors,  we  have  created  a  climate  of  a 
more  consistent  analytical  product,  with 
verification  and  validation  of  the  M&S  and 
data. 

But  has  there  been  a  paradigm  shift  in 
analysis  of  complex  warfare  as  there  has 
been  in  the  OR  cross  leveling  of  talent  and 
M&S?  The  answer  seems  to  be  yes,  but, 
again  it  is  tied  to  the  emerging  concepts 
hosted  in  model  development  efforts  like 
JWARS.  That  is,  the  measures  of  effec¬ 
tiveness  that  are  traced  from  the  basic  ana¬ 
lytical  question  to  the  applications  of  pro¬ 
gramming  techniques  must  also  be 
transparent  as  the  complexity  of  JWARS 
increases.  This  visibility  will  help  foster  a 
new  way  of  looking  at  what  analytical 
questions  to  ask  of  such  complex  combat 
models.  But,  to  better  understand  this,  we 
should  examine  the  basic  complexity  of 
JWARS. 

JWARS 

The  Joint  Warfare  System  (JWARS)  is 
an  analytic  model  of  combat  at  the  theater 
level.  Currently,  a  prototype  has  been  built 
and  tested  with  the  production  version 
under  way.  JWARS  will  be  a  closed-form 
simulation.  A  key  difference  between 
JWARS  and  the  similar  legacy  models  and 
simulations  of  the  past  30  years  is  that  it  is 
information-centric.  Legacy  models  and 
simulations  have  been  traditionally  attri¬ 
tion-centric.  This  means  that  intelligence 
fusion  is  central  to  the  operation  of  all  ele¬ 
ments  within  the  simulation,  which  gives 
the  user  community  an  analytical  tool  to 
analyze  the  complex  impacts  of  C^ISR 
(Command,  Control,  Communications, 
Computers,  Intelligence,  Surveillance,  and 
Reconnaissance)  to  a  complex  fit  of  joint 
mihtary  service  weapons  and  combat  sup¬ 
port  subsystems. 

The  model  is  a  closer  approximation  to 
real-world  joint  operations  than  any  previ¬ 
ous  similar  legacy  model  or  simulation. 
Some  of  the  joint  operational  or  mission 

(See  m Alts,  p,  20) 
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US  Army  Recruiting  Command,  Ft.  Knox,  KY;  and 

Dr.  Linda  C.  Malone 

Department  of  Industrial  Engineering,  University  of  Central  Florida 


Issues  concerning  the  verification  and 
validation  of  simulations  are  one  of  the 
classic  problems  facing  today’s  opera¬ 
tions  research  analyst.  There  are  several 
problems  with  existing  verification  and 
validation  techniques,  and  there  is  a  need 
for  a  structured  approach  to  verification 
and  validation.  The  purpose  of  this  article 
is  to  review  the  status  of  current  verifica¬ 
tion  and  validation  efforts  as  they  relate  to 
training  simulators,  and  to  propose  a 
methodology  for  effective  and  efficient 
verification  and  validation  efforts.  The 
methodology,  verification  and  validation 
through  interim  experimentation,  will  be 
presented  in  a  broad  sense.  This  article 
will  be  followed  by  another  article  that  pre¬ 
sents  a  specific  structured  approach  to 
interim  experimentation. 

One  major  problem  with  current  verifi¬ 
cation  and  validation  is  a  lack  of  organiza¬ 
tion  of  effort,  or  a  guiding  theory.  This 
results  in  experiments  that  are  poorly 
designed  that  cannot  provide  the  desired 
data,  that  answer  the  wrong  questions,  or 
that  duplicate  efforts.  Pace  (1995)  sug¬ 
gests  that  many  present  verification  and 
validation  efforts  become  mainly  judgment 
based.  This  condition  results  in  many 
efforts  for  verification  and  validation  con¬ 
taining  unwanted  variability.  Consequent¬ 
ly,  simulation  credibility  is  much  less  than 
desired.  A  structured  approach  is  needed 
to  ensure  experiments  are  designed  and 
organized  correctly  to  establish  quantifi¬ 
able  criteria  and  data  for  verification  and 
validation.  The  need  for  this  guiding  theo¬ 
ry  is  perhaps  even  more  crucial  for  the 
training  simulator  due  to  their  high  cost 
and  lengthy  development  cycle. 

Another  problem  associated  with  tradi¬ 
tional  verification  and  validation  efforts  is 
that  they  are  manpower  and  resource  inten¬ 
sive.  The  result  of  this  condition  is  that  the 
verification  and  validation  tests  are  often 
limited  in  scope  because  of  cost.  It  is  not 
unusual  in  some  systems  to  find  that  a  thor¬ 
ough  verification  and  validation  effort 
could  exceed  the  cost  of  the  product  devel¬ 
opment  (Gledhill,  1994).  The  cost  of 
experimentation  and  of  error  correction  is 


magnified  the  later  in  the  product  life  cycle 
the  experimentation  is  conducted.  A  struc¬ 
ture  guiding  experimentation  for  verifica¬ 
tion  and  validation  that  starts  early  in  the 
product  life  cycle  and  is  used  at  discrete 
intervals  will  help  alleviate  the  manpower 
requirements  and  the  cost  of  experimenta¬ 
tion  and/or  error  correction.  Several  small¬ 
er  scale  tests  conducted  through  the  life 
cycle  will  also  help  develop  a  better  prod¬ 
uct  than  one  large  test  at  the  end  of  devel¬ 
opment. 

Today’s  training  simulators  are  built 
upon  very  large  training  matrices  that  are 
quite  complex,  and  this  complexity  creates 
problems  for  verification  and  validation 
efforts.  As  technology  continues  to  move 
forward,  the  enumeration  of  all  possible 
training  scenarios  may  no  longer  be  feasi¬ 
ble  because  of  length  of  time  required  and 
cost.  If  an  experimenter  tries  to  enumerate 
all  of  the  possibilities,  a  hit  or  miss  proce¬ 
dure  will  result,  and  the  tester  will  not 
obtain  a  good  representation  of  all  training 
conditions  that  exist  in  the  simulator’s 
training  matrix.  Verification  and  valida¬ 
tion  tasks,  like  algorithm  analyses  that 
were  once  considered  optional,  are  now 
mandatory  to  verify  that  applications  inter¬ 
face  correctly  with  hardware.  The  result  of 
more  complex  systems  is  that  verification 
and  validation  should  be  used  earlier  and 
throughout  the  entire  life  cycle  (Dunham, 
1989). 

The  lack  of  a  standard  guiding  theory 
oriented  specifically  towards  training  simu¬ 
lators  is  another  basic  problem  associated 
with  verification  and  validation.  There  is  a 
great  deal  of  individual  information  avail¬ 
able  for  verification  and  validation  of  soft¬ 
ware,  for  verification  and  validation  of 
simulation  models,  and  for  system  testing 
for  specific  production  items.  Training 
simulators  involve  a  combination  of  each 
of  these  three  fields,  so  when  testing,  theo- 
ly  from  each  field  is  applicable.  A  set  of 
rules  or  a  structure  for  combining  the  theo¬ 
ry  would  be  very  helpful,  because  when 
each  are  integrated  separately,  problems 
can  easily  be  created,  or  critical  items  may 
be  overlooked. 


One  way  to  overcome  these  verification 
and  validation  problems  mentioned  earlier 
is  to  conduct  verification  and  validation  at 
discrete  points  in  the  simulator  develop¬ 
ment  cycle.  Before  the  opportunities  avail¬ 
able  for  interim  tests  can  be  presented,  ver- 
ification,  validation,  and  interim 
experiments  must  be  defined.  According 
to  Department  of  the  Army  Pamphlet  5-11 
(1993)  the  purpose  of  verification  of  the 
system  is  to  determine  that  the  system 
functions  as  it  was  originally  conceived, 
specified  and  designed,  and  that  it  meets 
the  needs  of  the  user  as  specified  in  the 
requirements  documents.  Department  of 
the  Army  Pamphlet  5-11  (1993)  states  the 
purpose  of  validation  of  the  simulator  is  to 
determine  the  extent  to  which  the  system 
accurately  represents  the  intended  real 
world  phenomenon  from  the  perspective  of 
the  customer  of  the  system.  The  interim 
tests  should  be  used  as  a  vehicle  for  verifi¬ 
cation  and  validation  of  the  system  under 
development.  One  of  the  greatest  benefits 
to  this  interim  testing  organization  is  that 
the  verification  and  validation  will  have  the 
input  and  approval  of  both  the  user  and  the 
producer.  Verification  and  validation  is 
one  of  the  most  resource  intensive  phases 
in  the  development  of  a  training  simulation 
system.  It  is  also  the  most  crucial  to  the 
success  of  a  program.  Verification  and 
validation  efforts  are  a  part  of  the  simula¬ 
tor’s  product  development  cycle  that  occur 
between  the  development  of  a  prototype 
system  and  the  system’s  production.  Veri¬ 
fication  and  validation  efforts  can  be 
extended  to  a  system  after  it  is  fielded. 
However,  the  approach  used  in  this  discus¬ 
sion  will  strive  for  identification  and  reso¬ 
lution  of  issues  prior  to  production.  The 
verification  and  validation  will  be  conduct¬ 
ed  through  the  use  of  interim  experimenta¬ 
tion. 

An  interim  experiment  is  an  exercise 
involving  both  the  producer  and  the  user  of 
a  training  simulator.  It  is  conducted 
throughout  the  product  development  cycle, 
with  a  purpose  of  ensuring  that  the  system 
under  development  is  meeting  some 
agreed  upon  specifications  or  require- 
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meats.  The  exercises,  or  testing,  should 
begin  early  in  the  product  life  cycle  and 
should  be  conducted  at  stages  throughout 
the  product  development  cycle.  The  pro¬ 
ducer  is  the  organization  contractually 
responsible  for  the  production  of  the  device 
against  detailed  requirements  or  specifica¬ 
tions.  The  user  is  the  agency  that  needs  the 
system  under  development  to  fill  a  specific 
training  requirement.  The  user  is  the  origi¬ 
nator  of  the  statement  of  requirements  or 
specifications  documents.  There  may  be 
several  organizations  and  players  struc¬ 
tured  within  the  user  agency.  There  may 
be  a  separate  group  detailed  with  the  acqui¬ 
sition  of  the  training  system,  such  as  the 
Simulation,  Training  and  Instrumentation 
Command  (STRICOM).  There  will  be  a 
specific  training  audience  within  the  user 
group,  like  the  tactical  units  that  will  even¬ 
tually  train  on  the  simulators.  There  will 
also  be  several  different  types  of  subject 
matter  experts  within  the  user  organization. 
These  experts  will  be  called  upon  at  differ¬ 
ent  intervals  to  ensure  the  training  simula¬ 
tor  is  behaving  like  the  modeled  system. 
Some  of  these  experts  may  be  experts  in 
training  devices  or  simulators.  An  exam¬ 
ple  of  this  type  of  expert  are  the  instruc¬ 
tor/operators  (I/O)  for  the  simulators  at  the 
US  Army  Armor  School.  Some  may  be 
field  matter  experts  in  the  area  for  which 
the  training  simulator  is  being  developed, 
like  the  soldiers  who  have  spent  the  major¬ 
ity  of  their  careers  on  the  actual  equipment 
being  simulated.  All  of  these  participants 
will  serve  various  important  roles  in  the 
interim  experiment  process.  The  interim 
experiment  can  be  conducted  at  different 
levels  for  different  purposes.  It  can  be  a 
demonstration  of  the  progress  of  the  cur¬ 
rent  development  of  a  system,  or  it  can  be  a 
concurrent  engineering  effort  attempting  to 
curtail  changes  late  in  the  product  life 
cycle.  It  can  be  a  one  time  exercise  for  a 
prototype  system  that  is  used  as  a  produc¬ 
tion  milestone  or  it  can  be  a  sequential  test 
following  the  test,  fix,  test  methodology. 
The  interim  experiment  can  be  many 
things  to  many  products.  There  is  no  exact 
solution  for  organization  of  every  system. 

Before  a  quantifiable  structured 
approach  to  verification  and  validation  of 
simulators  can  be  developed,  a  process  for 
organization  is  required.  Component  level 
is  the  first  level  analyzed.  The  producer 
team  should  develop  a  list  of  all  major 
components  of  the  training  simulator. 
From  this  list,  both  teams  will  jointly  deter¬ 


mine  which  components  are  critical  and 
warrant  the  participation  of  both  the  user 
and  producer  in  verification  and  validation 
through  the  means  of  an  interim  experi¬ 
ment.. 

Each  team  should  develop  a  list  of  per¬ 
sonnel  required  for  participation  in  the 
component  level  of  testing.  Each  team 
leader  is  responsible  for  his  own  group. 
Lists  should  be  agreed  upon  by  both  teams, 
so  each  team  is  happy  with  representation 
provided  by  the  other  group.  From  the 
user  team,  potential  personnel  include 
those  who  can  readily  check  software  code, 
methodology  of  the  code,  structural  and 
static  validation,  and  other  details  of  the 
component  nature.  From  the  producer 
team,  members  who  can  readily  explain 
and  correct  code  errors,  structural  errors,  or 
defend  logic  of  the  methodology  of  each 
component  would  be  ideal  candidates. 
This  list  will  form  the  interim  experimental 
exercise  team  for  the  component  level. 
The  producer  and  user  teams  should  also 
agree  on  the  role  and  participation  of  the 
independent  review  team  at  the  component 
level. 

Given  the  list  of  critical  components  to 
check  and  the  requirements  and  availability 
of  critical  personnel  on  each  team,  the  test¬ 
ing  team  chief  should  determine  how  many 
exercises  are  required  to  capture  all  of  the 
required  items.  The  team  chief  should  pre¬ 
sent  this  data  to  the  user  and  producer  lead¬ 
ership  for  approval.  This  number  is  the 
number  of  experiments  conducted  at  the 
first  level. 

The  following  example  of  a  tank  simu¬ 
lator  should  clarify  component  level  issues. 
Components  involved  are  the  data  base, 
used  for  storing  information  files;  the  gun¬ 
ner’s  power  control  handles,  used  for  firing 
the  tank;  the  tank  commander’s  indepen¬ 
dent  thermal  viewer,  used  for  acquiring 
and  engaging  targets;  and  the  instructor/ 
trainer’s  display  panel,  used  for  monitoring 
the  training  session.  There  will  always  be 
a  large  number  of  individual  components, 
but  all  are  not  critical.  Testing  the  gun¬ 
ner’s  control  handles  is  probably  not  a  crit¬ 
ical  item  for  all  members  of  the  testing 
team,  especially  if  the  part  is  relatively 
common.  If  the  commander’s  independent 
thermal  viewer  is  a  new  addition  to  the 
simulator,  and  has  never  been  seen  before 
by  the  user  on  a  simulator,  then  it  could  be 
considered  critical  and  worth  detailed  test¬ 
ing.  The  next  level  to  consider  is  the  inte¬ 
gration  level. 


Integration  level  analysis  is  done  after 
the  component  level.  The  producer  team 
should  identify  each  phase  of  integration  of 
components  from  individual  components 
through  complete  integration.  From  this 
list  both  the  user  and  producer  team  leaders 
should  decide  which  phases  are  significant 
enough  to  warrant  detailed  experimenta¬ 
tion.  The  testing  team  chief,  again, 
resolves  conflicts  and  offers  advice. 

The  user  team  leader  should  identify 
personnel  who  are  experts  needed  to  com¬ 
pare  integration  and  system  response  given 
specified  inputs  with  responses  of  real  sys¬ 
tems.  The  list  should  include  personnel 
familiar  enough  with  the  simulator  to  be 
able  to  explain  accepted  trade-offs.  From 
the  producer  side,  the  team  leader  should 
include  experts  able  to  readily  correct  inter¬ 
face  errors,  and  debug  systems  given  indi¬ 
cator  errors. 

The  testing  team  chief  needs  to  compare 
the  number  of  critical  phases  of  integration 
to  the  number  and  quantity  of  required  per¬ 
sonnel  from  all  sides  in  order  to  determine 
the  number  of  required  exercises  at  the 
integration  level. 

Examples  of  integration  on  a  flight  sim¬ 
ulator  are  combining  the  flight  control  sys¬ 
tem  and  the  gunnery/armament  system  in 
an  Apache  flight  simulator.  The  specific 
code  issues  of  the  computer  interface  may 
be  of  less  interest  to  the  user  than  the  end 
result  of  what  happens  when  a  trigger  is 
pulled  in  the  gunner’s  station. 

System  level  requirements  are  checked 
after  integration  level.  From  the  available 
training  menu  and  the  detailed  system 
requirements  and  specifications,  each  team 
leader  should  determine  which  training 
components  require  detailed  testing  that 
involves  both  the  user  and  producer.  The 
independent  review  team  should  be  includ¬ 
ed  in  this  decision. 

Each  team  leader  is  responsible  for 
deciding  which  experts  and  participants 
wiU  be  needed  at  each  level  of  the  training 
menu  to  effectively  compare  the  simulator 
with  the  real  system,  identify  errors  at  the 
source  and  implement  corrections,  or  dis¬ 
cuss  trade-offs  accepted  by  all  participants. 
The  chief  of  testing  again  plays  a  large  role 
in  resolving  problems.  The  testing  chief 
will  decide  how  many  experiments  should 
be  conducted  based  on  the  list  of  critical 
training  menu  checks  and  required  person¬ 
nel. 

(See  TRAINING  SIMULATORS,  p.  24) 
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John  Key  Walker,  Jr.,  78,  of  Jensen 
Beach,  Florida  and  Silver  Spring,  Mary¬ 
land,  died  on  29  January  1998,  in  Stuart, 
Florida  of  complications  following  lung 
surgery. 

Jack  graduated  from  Bedford  (Virginia) 
High  School;  received  a  BS  degree  in  1941 
from  Virginia  Polytechnic  Institute  and  an 
MS  in  Management  Science  from  George 
Washington  University  in  1971. 

He  entered  the  US  Army  in  1941  with 
an  ROTC  commission  as  2nd  Lieutenant. 
His  commands  included  the  486th  Anti- 
Aircraft  Artillery  Battalion  of  the  3rd 
Armored  Division  (World  War  II)  and  3rd 
Brigade,  2nd  Infantry  Division  (Ft.  Ben- 
ning,  GA).  Tours  of  duty  included  Head¬ 
quarters,  Vll  Corps  (Germany),  Command 
and  General  Staff  College  (Ft.  Leaven¬ 
worth,  Kansas),  the  Army  War  College, 
(Carlisle  Barracks,  Pennsylvania),  Allied 
Forces  North  (Oslo,  Norway),  Supreme 
Headquarters  Allied  Powers  Europe 
(Paris,  France),  Army  General  Staff,  Office 
of  the  Secretary  of  Defense  (Pentagon)  and 


ARVN  5th  Infantry  Division  (Viet  Nam). 

Decorations  include  the  Silver  Star  with 
Oak  Leaf  Cluster,  Distinguished  Flying 
Cross,  Bronze  Star  Medal  for  Valor  and 
Legion  of  Merit  with  Oak  Leaf  Cluster. 

In  1969  he  retired  as  Colonel,  then 


served  as  a  military  research  analyst  with 
The  RAND  Corporation,  Washington,  DC, 
from  1969-95. 

Jack  is  survived  by  his  wife,  Toni,  three 
daughters,  three  step-children,  six  grand¬ 
children,  four  step- grandchildren,  three 
nieces  and  one  nephew. 

A  memorial  service  was  held  in  Jensen 
Beach  on  31  January  1998.  Internment 
will  be  at  Arlington  National  Cemetery  at  a 
later  date.  The  family  requests  that  contri¬ 
butions  in  Jack's  name  be  sent  to  National 
D-Day  Foundation,  P.O.  Box  77,  Bedford, 
V A  24523. 

Jack  was  the  PHALANX  Editor  Emeri¬ 
tus,  was  elected  one  of  the  first  Fellows  of 
MORS  in  1989,  was  a  Past  President  of 
MORS  and  served  the  MORS  community 
since  1969.  He  will  be  greatly  missed  by  us 
all. 

The  following  are  tributes  to  Jack  from 
some  of  his  fellow  MORS  PHELLOWS. 

—  Natalie  Addison 


A  Tribute  to  Jack  from  Wayne  Hughes 


Wayne  Hughes,  FS 

The  Board  created  the  MORS  Fel¬ 
lows,  but  Jack  Walker,  FS  creat¬ 
ed  the  PHALA  Phellows.  His 
column,  so  full  of  wit,  says  worlds  about 
Jack’s  pixie  imagination  and  devotion  to 
service.  If  he  could  write  his  own  memori- 
am  doubtless  he  would  wish  us  to  know 
that  even  Phine  Phellows  are  Phated  but 
Phortunate  to  move  to  new  and  Phacinat- 
ing  Phases. 

Jack  Walker,  FS  is  the  very  image  of 
selfless  service  —  three  times  over.  In  a 
full  Army  career  of  27  years  he  rose  from 
2nd  Lieutenant  to  Colonel,  was  in  action  in 
three  wars,  and  seems  to  have  been  every¬ 
where  and  done  everything  (an  Army  offi¬ 
cer  with  a  Distinguished  Flying  Cross?!). 
Then  he  started  over  again.  At  RAND  he 
charted  another  career  of  more  than  20 
years,  this  time  in  operations  analysis, 
which  was  equally  notable  for  getting  his 
job  done  with  quiet  competence. 


Paralleling  his  RAND  career  was  a  third 
one  with  the  Military  Operations  Research 
Society  —  maybe  not  full  time,  it’s  true, 
but  performed  without  a  whimper  or  a 
dime  of  compensation  —  and  it  lasted 
almost  another  30  years.  You’ll  find  his 
column  in  the  latest  PHALANX,  December 
1997.  So  there  you  are:  by  my  count,  80 
career  years,  in  series  and  in  parallel,  and 
as  far  as  I  know  all  80  of  them  devoted  to 
positive  thinking,  morale  building,  and 
unostentatious,  responsible  achievement 
I  came  upon  MORS  at  almost  the  same 
time  as  Jack.  It  was  around  1970.  He  led 
me  by  only  a  couple  of  years,  but  in  the 
eyes  of  young  Commander  Hughes,  Jack 
was  already  one  of  the  venerables.  Noth¬ 
ing  I  saw  him  do  ever  changed  my  mind. 
One  example.  When  I  was  MORS  Presi¬ 
dent  I  had  (as  I  now  recall)  only  one  burn¬ 
ing  problem.  Jack  had  been  editor  of  PHA¬ 
LANX  for  what  was  already  a  very  long 


time.  Furthermore  he  had  already  served 
MORS  in  every  other  imaginable  way,  and 
he  had  for  several  years  been  dropping 
hints,  subtle  or  blunt,  that  we  needed  a  new 
editor.  At  Symposia  and  Board  Meetings  I 
winced  every  time  Jack’s  camera  flashed 
or  he  presented  one  of  those  albums  of 
photos  that  were  half  esteem  and  half 
hatchet-job.  My  term  ended  with  no 
replacement  for  him.  I  had  done  a  lot  of 
stroking  so  that  Jack  wouldn’t  quit  and  felt 
quite  proud  of  my  achievement.  Such 
foolishness!  I  know  now  that  Jack  never 
walked  away  from  anything  in  his  life.  He 
wouldn’t  have  quit  if  I’d  spit  on  him. 

And  the  ranks  of  the  Old  Guard  with 
memories  of  how  it  really  was  in  World 
War  II  continues  to  shrink.  Jack  Walker 
did  his  share  and  a  lot  more  to  bring  on 
three  new  generations,  in  the  Army,  in 
RAND,  and  in  MORS.  He  is  one  Phenom¬ 
enal  Phellow.  O 
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How  I  Remember  Jack  Walker 

Steve  Murtaugh,  FS 


How  would  such  a  good  friend  as  Jack 
Walker,  FS  want  us  to  remember 
him?  This  is  the  thought  that  comes 
to  mind.  As  I  prepare  to  write  this  portion  of  a 
tribute  to  Jack,  for  sure,  each  of  us  companion 
Fellows  will  have  different  approaches  to  what 
we  write  for  this  occasion,  for  Jack  Walker 
was  many  things  to  each  of  us.  But,  to  one  and 
all.  Jack  was  first  a  true  friend  (and  that  is  a 
rare  possession  in  today’s  world).  This  is 
exactiy  what  Jack  was,  and  is  a  prime  reason 
why  so  many  of  us  valued  our  association  with 
him. 

Jack  went  beyond  mere  friendship  —  he 
worked  at  it  actively.  For  example,  back  in  the 
early  60’ s,  when  I  was  a  newly  elected  MORS 
Director,  Jack  and  Clay  Thomas,  FS  took  me 
in  tow  and  taught  me  how  to  contribute  on  the 
Board  for  my  MORS  career,  always  in  true, 
caring  friendship. 

We  all  knew  Jack  as  editor  of  the  PHA¬ 
LANX  —  a  job  he  did  admirably  for  many 
years  —  but  he  did  so  much  more  for  the 
PHALANX.  He  led  the  effort  to  expand  P/M- 
LANX  from  a  newsletter  to  a  professional  pub¬ 
lication,  expanding  its  size,  attracting  worth¬ 
while  authors,  and  instilling  many  innovative 
features.  Jack  initiated  the  series  of  P/MLAAX" 
articles  in  which  a  different  Fellow  is  written 
up  in  each  issue  for  all  the  membership  to  read. 
This  has  been  a  richly  successful  feature  in 
PHALANX;  Jack’s  approach  to  writing  these 
highly  readable  articles  will  be  missed.  We  are 
all  proud  of  the  excellence  Jack  brought  to 
PHALANX  —  the  showcase  of  our  Society’s 
activities  and  personages. 

Before  PHALANX,  Jack  was  helping  to 
bond  our  Society  in  other  ways.  He  chaired 
the  33^^  Symposium  and  was  elected  Presi¬ 
dent  shortly  thereafter.  When  Vance  Wanner, 
Executive  Secretary  of  MORS,  died  unexpect¬ 
edly  some  years  ago,  Jack  moved  into  the 
MORS  Office,  and  with  the  early  staff,  kept 
the  Society  functioning  until  a  new  Executive 
Secretary  could  be  found.  Jack  made  the  time 
to  do  this  even  though  he  was  working  full 
time  at  the  RAND  Corporation.  When  Ed 
Napier,  FS  agreed  to  be  the  new  Executive 
Secretary,  Jack  stayed  on  to  work  with  Ed  and 
make  the  transition  smooth.  When  we  were 
developing  the  Society’s  first  Code  of  Ethics, 
Jack  came  forward  with  all  kinds  of  ethics 
materials  from  MORS  and  ORSA,  developed 


when  Jack  chaired  an  ethics  working  group  in 
early  MORS  days. 

When  five  of  us  were  selected  as  the  first 
MORS  Fellows,  we  wrestled  with  how  the 
Fellows  could  continue  to  contribute  as  a 
group  in  some  meaningful  way  to  the  Society. 
Once  again.  Jack  was  there  with  an  abun¬ 
dance  of  good  ideas  to  submit  to  the  Board, 
and  when  we  were  charged  by  the  Council 
with  identifying  those  early  MORS  workers 
who  would  be  Fellows  candidates.  Jack  was 
there  with  suggestions  and  supporting  guide¬ 
lines. 

These  are  but  a  few  of  the  ways  in  which  I 
shall  remember  Jack  Walker.  I  have  one 
more  memory  to  share  with  you  —  and  that  is 
of  Jack  and  his  lovely  wife  Toni  and  their 
presence  at  each  Symposium  over  many 
years.  They  always  invited  me  to  join  them  at 


Tribute  to  Jack 

Clay  Thomas,  FS 

I  can  think  of  no  one  whose  loss  will  be 
mourned  by  more  of  us  “MORSians,” 
than  Jack  Walker,  FS.  MORS,  of 
course,  is  only  one  of  the  institutions  that  Jack 
enriched  with  his  friendly  help,  dedicated 
industry,  talented  insights,  and  unparalleled 
sense  of  humor.  But  I  knew  him  best  as  a 
MORSian  who  outstandingly  epitomized  so 
many  facets  of  MORS,  and  whom  so  many  of 
us  remember  as  a  warm,  highly  valued,  and 
vital  friend. 

Slightly  over  a  quarter  of  a  century  ago  I 
was  to  chair  the  30^^  MORS  Symposium. 
Feeling  a  bit  overwhelmed,  I  sought  outstand¬ 
ing  MORSians  to  fill  the  critical  positions,  and 
the  first  one  I  turned  to  was  Jack  Walker,  Not 
only  did  he  agree  to  be  General  Sessions  coor¬ 
dinator,  the  key  position  of  that  time,  but  he 
accepted  in  that  marvelously  gracious  way  as 
if  I  were  doing  him  a  favor!  And  his  top- 
notch  performance  was  what  we  all  came  to 
expect  as  a  matter  of  course,  whatever  his  role. 

In  that  period  Jack  came  to  be  known  in 
MORS  for  his  innovative  work  on  Ethics  and 
Professional  Practice.  It  was  a  subject  of  inter¬ 
est  in  other  professional  societies  as  well,  but 
it  was  Jack  who  sensed  the  particular  impor¬ 
tance  of  the  subject  for  a  society  with  military 


the  Symposia  banquets.  I  especially  recall  the 
director’s  dinner  at  Annapolis  in  1990  when 
Jack  was  toasted  the  evening  before  receiving 
the  prestigious  Wanner  Award  —  even  then 
Toni  and  Jack  invited  me  to  sit  with  them  at 
dinner,  always  willing  to  share  their  good 
times. 

I  know  I  am  a  better  person  and,  I  hope,  a 
better  MORS  contributor,  because  Jack  and  I 
not  only  crossed  paths,  but  because  Jack 
chose  to  walk  with  me  at  significant  times  in 
my  MORS  career,  always  there  to  help  and  to 
encourage.  The  records  show  that  Jack  Walk¬ 
er  was  one  the  of  the  first  MORS  Fellows,  a 
Wanner  Award  recipient,  and  PHALANX  edi¬ 
tor  emeritus,  but  in  my  heart,  Jack  will  always 
be  remembered  as  a  true  friend,  and  in  the 
hearts  of  many  others.  I’m  sure,  a  thoughtful 
counselor,  and  an  ever  faithful  supporter.  O 


sponsorship  that  sought  to  enhance  the  quality 
of  operations  research  studies  in  an  area  criti¬ 
cal  to  national  defense  policy.  Jack’s  pioneer¬ 
ing  work  paved  the  way  for  MORS’  continu¬ 
ous  and  successful  involvement  in  this  area, 
and  its  reputation  for  high  ethical  standards. 

It  was  as  PHALANX  Editor,  however,  that 
Jack  came  to  be  so  widely  known,  not  only 
throughout  MORS,  but  in  other  large  OR  pro¬ 
fessional  societies  as  well.  The  original  deci¬ 
sion  of  the  MORS  Board  of  Directors  in  1973 
to  accept  the  invitation  to  become  a  co-spon- 
sor  of  the  PHALANX  Newsletter  (from  ORSA 
MAS)  was  far  from  popular.  One  Director 
resigned  to  protest  what  he  thought  was  an 
unfortunate  mistake!  Of  all  those  whose  hard 
work  transformed  PHALANX  from  a  small 
Newsletter  to  the  large  and  successful  Bulletin 
of  today,  Jack  stands  out  for  his  dozen  years  as 
Editor  (1979-1991).  His  introduction  of  new 
departments,  active  search  for  topical  OR  sto¬ 
ries,  irrepressible  sense  of  humor,  and,  of 
course,  his  careful  editing,  were  what  set  the 
PHALANX  tone  and  made  every  issue  reward¬ 
ing. 

I  can  picture  Jack  now,  looking  over  our 
shoulders,  commenting  that  this  issue  needs  a 
little  more  work.  O 
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Still  Need  a  Reason  to  Go  to  Monterey? 

RADM  Pierce  Johnson,  Program  Chair 

Ted  Smyth,  Plenaiy/Special  Sessions  Coordinator 

CDR  Ron  Brown,  Site  Coordinator 


If  you  haven’t  made  plans  to  attend  the 
66th  MORSS,  it  is  time  to  begin!  The 
66th  Military  Operations  Research 
Society  Symposium  will  be  held  at  the 
Naval  Postgraduate  School  (NPS)  in  Mon¬ 
terey,  California,  on  23-25  June  1998,  with 
a  theme  of  Preparing  for  Military  Opera¬ 
tions  Research  in  the  27  st  Century.  Early 
registration  ends  in  just  a  few  weeks  (on 
April  19th).  If  you  haven’t  decided  on 
attending  yet,  here  are  a  couple  of  reasons 
why  you  might  want  to  consider  coming:  a 
quality  keynote  speaker;  applicable,  for¬ 
ward-thinking  special  sessions  from  our 
senior  analysts  and  our  senior  leadership; 
and  the  overall  magnificence  of  the  Mon¬ 
terey  Peninsula. 

Plenary  Session 

The  66th  MORSS  is  honored  to  have  as 
its  Keynote  Speaker,  Dr.  William  J. 
Perry,  the  former  Secretary  of  Defense. 
Dr.  Perry  is  currently  a  Senior  Fellow  at 
Stanford  University  where  he  holds  the 
Michael  and  Barbara  Barberian  Professor¬ 
ship.  Dr.  Perry’s  Keynote  Address  will  be 
delivered  during  the  Plenary  Session 
scheduled  for  0830-1000,  Tuesday,  23 
June  1998. 

Special  Sessions 

During  the  66th  MORSS,  Special  Ses¬ 
sions  will  be  conducted  daily  from  1530- 
1700.  A  wide  variety  of  activities  are 
scheduled  for  each  of  the  Special  Sessions. 
The  Special  Session  scheduled  for  Tues¬ 
day,  23  June  will  focus  on  the  subject  of 
“Leadership  in  an  Information  Dominant 
Battle.”  The  purpose  of  this  Special  Ses¬ 
sion  is  to  challenge  the  audience  to  recon¬ 
sider  the  relative  importance  of  technology 
and  leadership  in  an  information  dominant 
military.  It  is  anticipated  that  participants 
will  address  the  relationship  that  exists 
between  technology  and  leadership  and  the 
synergy  that  may  be  derived  by  effectively 
nurturing  and  developing  the  two.  Other 
activities  scheduled  during  the  Tuesday 


Special  session  include  reports  from  sever¬ 
al  Mini  Symposia  to  include  the  September 
1997  Mini  Symposium  on  “Complexity 
and  Warfare  Analyses”  and  the  planned 
April  1998  Mini  Symposium  on  “Analysis 
Requirements  for  the  Next  QDR.”  Prize 
Paper  Presentations  are  also  scheduled  for 
the  Tuesday  Special  Session. 

The  Wednesday,  24  June  Special  Ses¬ 
sion  will  focus  on  the  issue  of  “Validation 
Methodologies.”  It  is  anticipated  that  sev¬ 
eral  case  studies  will  be  presented  and  dis¬ 
cussed  by  a  select  number  of  panelists. 
Panelists  will  briefly  describe  the  model  in 
question,  the  techniques  used  in  the  valida¬ 
tion  effort,  and  emphasize  the  lessons 
learned  from  the  validation  effort.  Other 
activities  currently  planned  for  the 
Wednesday  Special  Session  include  a 
report  from  the  December  1997  SIMTECH 


2007  Workshop  and  the  popular  Junior/ 
Senior  Analyst  session. 

The  Thursday,  25  June  Special  Session 
will  focus  on  a  Navy  3-Star  panel  discus¬ 
sion  on  the  subject  of  “Preparing  Today’s 
Operations  Research  Analysts  for  the  Next 
Century.”  Invited  panelists  include  Vice 
Admirals  Lautenbacher,  Redd, 
Cebrowski,  Oliver,  and  Tracey.  All  of 
these  invited  officers  have  either  an  opera¬ 
tions  research  background  and/or  consider¬ 
able  experience  and  knowledge  of  the  con¬ 
tributions  provided  by  the  operations 
analysis  community  in  support  of  both 
operational  and  program  requirements. 
Other  activities  scheduled  for  this  third  and 
final  Special  Session  include  reports  from 
the  Spring  1998  Education  Colloquium 
and  the  January  1998  “DoD  Infrastructure” 
Mini-Symposia. 
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History  of  the  Naval 
Postgraduate  School 

The  Naval  Postgraduate  School  began 
to  evolve  when  the  idea  for  a  graduate  edu¬ 
cation  program  for  naval  officers  first 
emerged  in  the  late  19th  century.  Initially, 
the  concept  found  few  advocates.  With 
Marconi’s  invention  of  the  “wireless”  in 
1901,  the  Wright  brothers’  flight  in  1903, 
and  the  global  trek  of  the  steam-powered 
White  Fleet  from  1907  to  1909,  the  con¬ 
cept  of  advanced  education  for  US  Naval 
Officers  gained  support. 

On  9  June  1909,  less  than  four  months 
after  the  completion  of  the  record-setting 
world  cruise  of  the  Great  White  Reet,  Sec¬ 
retary  of  the  Navy  George  von  L.  Meyer 
signed  General  Order  No.  27,  establishing 
a  school  of  marine  engineering  at  Annapo- 
hs.  This  small  program,  consisting  of  10 
officer  students  and  two  Navy  instructors, 
would  later  become  today’s  Naval  Post¬ 
graduate  School.  The  Navy  secretary’s 
order  placed  the  fledgling  school  under  the 
direction  of  the  Naval  Academy  superin¬ 
tendent,  who  was  charged  with  “securing 
ample  use  of  the  educational  plant  of  the 
Naval  Academy  for  students  and  instruc¬ 
tors  of  the  school  without  interfering  with 
the  instruction  of  midshipmen.”  This 
translated  into  two  attic  rooms  being  set 
aside  for  classroom  and  laboratory  space 
for  the  school. 

Within  three  years,  Meyer  agreed  to  a 
proposal  to  change  the  school.  On  31 
October  1912,  he  signed  Navy  General 
Order  No.  233  which  changed  the  school’s 
name  to  the  Postgraduate  Department  of 
the  Naval  Academy  and  added  courses  of 
study  in  ordnance  and  gunnery,  electrical 
engineering,  radiotelegraphy,  naval  con¬ 
struction,  and  civil  engineering  to  the  origi¬ 
nal  program  in  marine  engineering. 
Enrollment  increased  to  25  officers  who 
finished  their  academic  programs  at  civil¬ 
ian  institutions  such  as  Yale,  Harvard,  the 
Massachusetts  Institute  of  Technology  and 
The  Johns  Hopkins  and  Columbia  Univer¬ 
sities  after  attending  the  school. 

During  World  War  II,  Fleet  Admiral 
Ernest  King,  Chief  of  Naval  Operations 
and  Commander-in-Chief  of  both  the 
Atlantic  and  Pacific  fleets,  estabhshed  the 
Pye  commission  to  review  the  role  of  grad¬ 
uate  education  in  the  Navy,  This  group’s 


recommendations  were  regarded  highly 
within  the  Navy  and  Congress.  In  1945, 
Congress  passed  legislation  to  make  the 
school  a  fully  accredited,  degree-granting 
graduate  institution.  Two  years  later.  Con¬ 
gress  adopted  legislation  authorizing  the 
purchase  of  an  independent  campus  for  the 
school. 

After  examining  25  sites  nationwide,  a 
post-war  review  team  recommended  the 
old  Del  Monte  Hotel  as  the  Postgraduate 
School’s  new  home.  The  Navy  had  first 
come  to  Monterey  in  early  1943  when  it 
leased  the  Del  Monte  Hotel  for  a  preflight 
training  school,  which  was  subsequently 
used  for  other  training  programs.  Negotia¬ 
tions  with  the  Del  Monte  Properties  Com¬ 
pany  led  to  the  purchase  of  the  hotel  and 
627  acres  of  surrounding  land  for  $2.13 
million. 

In  December  1951,  in  a  move  virtually 
unparalleled  in  the  history  of  academe,  the 
Postgraduate  School  moved  lock,  stock 
and  wind  tunnel  across  the  nation,  estab¬ 
lishing  its  current  campus  in  Monterey. 
The  coast-to-coast  move  involved  500  stu¬ 
dents,  about  100  faculty  and  staff  and  thou¬ 
sands  of  pounds  of  books  and  research 
equipment  and  pumped  new  vitality  into 
the  Navy’s  efforts  to  advance  naval  science 
and  technology. 

Today’s  Postgraduate  School  is  an  aca¬ 
demic  institution  that  continues  to  empha¬ 
size  study  and  research  programs  relevant 
to  the  Navy’s  interests,  as  well  as  the  inter¬ 
ests  of  other  arms  of  the  Department  of 
Defense,  Its  40  programs  of  study,  specifi¬ 
cally  designed  to  accommodate  the  unique 
requirements  of  the  military,  provide 
advanced  degrees  in  National  Security 
Affairs,  Aeronautical  Engineering,  Apphed 
Mathematics,  Applied  Physics,  Applied 
Science,  Astronautical  Engineering,  Com¬ 
puter  Science,  Defense  Analysis,  Electrical 
Engineering,  Engineering  Acoustics,  Engi¬ 
neering  Science,  Information  Technology 
Management,  International  Resource  Plan¬ 
ning  and  Management,  Management, 
Materials  Science  and  Engineering, 
Mechanical  Engineering,  Meteorology, 
Meteorology  and  Physical  Oceanography, 
Modeling,  Virtual  Environments  and  Sim¬ 
ulation,  Operations  Research,  Physical 
Oceanography,  Physics,  Systems  Engi¬ 
neering,  and  Systems  Technology.  The 


student  population  has  grown  to  1,800  and 
includes  officers  from  all  five  US  uni¬ 
formed  services,  the  services  of  more  than 
25  allied  nations,  and  a  small  number  of 
civihan  employees  of  the  US  government. 
The  campus  covers  627  acres  of  land  and 
houses  state-of-the-art  laboratories,  numer¬ 
ous  academic  buildings,  a  great  library, 
government  housing  and  impressive  recre¬ 
ational  facilities.  Selection  of  officers  for 
fully  funded  graduate  education  is  based 
upon  outstanding  professional  performance 
as  an  officer,  promotion  potential  and  a 
strong  academic  background.  The  faculty, 
composed  primarily  of  civilians,  is  drawn 
from  a  broad  diversity  of  educational  insti¬ 
tutions  and  represents  a  prestigious  collec¬ 
tion  of  scholars.  Faculty/student  interac¬ 
tion  is  high,  with  every  class  taught  directly 
by  a  faculty  member  —  over  99%  have  a 
Ph.D. 

Monterey  Peninsula 

The  Postgraduate  School’s  setting  on 
the  magnificent  Monterey  Peninsula  is  one 
of  the  most  diverse  and  beautiful  areas  in 
Northern  California,  Often  called  the 
greatest  meeting  of  land,  sea  and  sky,  this 
beautiful  seaside  community  combines  all 
the  charm  of  small  town  America  with  an 
endless  variety  of  recreational  and  cultural 
activities.  Monterey’s  rich  history,  the 
saga  of  Cahfomia’s  Mission  Trail,  historic 
Fisherman’s  Wharf  and  Cannery  Row,  17 
world-class  golf  courses,  the  world’s  best 
aquarium,  a  unique  variety  of  shops  and 
galleries  and  a  spectacular  assortment  of 
parks  and  natural  areas  combine  to  provide 
a  truly  unrivaled  vacation  or  business  desti¬ 
nation. 

Monterey  itself  is  but  one  of  several 
attractions  of  the  Monterey  Bay  area.  Visi¬ 
tors  enjoy  strolling  the  streets  of  Stein¬ 
beck’s  Carmery  Row,  absorbing  the  histor¬ 
ical  and  cultural  past  of  California’s  first 
state  capital.  Monterey  also  features  Fish¬ 
erman’s  Wharf  and  the  Monterey  Bay 
Aquarium  in  addition  to  its  array  of  fine 
restaurants,  boutiques,  galleries,  inns  and 
other  recreational  opportunities.  It  is  host 
to  a  wide  variety  of  festivals  and  world 
class  events  including  The  Monterey  Jazz 
Festival,  The  Monterey  Blues  Festival 

(See  SYMPOSIUM, 
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(which  is  scheduled  the  weekend  after  the 
symposium),  Cherries  Jubilee,  The  Great 
Monterey  Squid  Festival  and  Monterey 
Wine  Festival,  as  well  as  major  automobile 
and  motorcycle  races  at  nearby  Laguna 
Seca  Raceway. 

Nearby,  Carmel-by-the-Sea  had  already 
achieved  its  international  reputation  as  an 
artists’  colony  by  the  1920’s.  Among  its 
biggest  attractions  today  are  Point  Lobos, 
Carmel  and  Lagoon  Beaches,  Ocean 
Avenue,  The  Tor  House,  Carmel  Mission, 
and  a  variety  of  art  and  photography  gal¬ 
leries  in  its  downtown  shopping  district. 
You  won’t  find  many  sidewalks,  street¬ 
lights,  neon  signs  or  mailing  addresses,  but 
you  will  find  specialty  shops,  boutiques,  art 
and  photography  galleries  and  great  restau¬ 
rants.  Pacific  Grove,  another  local  attrac¬ 
tion,  is  famous  for  its  thriving  population 
of  Monarch  butterflies,  A  walk  through  its 
residential  neighborhoods  reveals  many 
well-preserved,  turn  of  the  century,  Victo¬ 
rian  homes.  History  buffs  can  visit  the 
Point  Pinos  Lighthouse,  the  oldest  operat¬ 
ing  facility  of  its  kind  on  the  California 
coast.  The  Monterey  Bay  recreational  trail 
passes  by  Lover’s  Point,  offering  many 
opportunities  to  view  sea  otters,  sea  lions 
and  occasionally  passing  whales.  Yet 


JWARS 

(continued  from  p.  13) 

areas  modeled  are: 

•  C^ISR 

•  Intra  and  inter-theater 
transportation 

•  Logistics 

•  Firepower 

•  Land,  air  and  sea  operations 

However,  it  is  the  C^ISR  that  is  the 
infrastructure  of  all  simulated  systems  and 
activities.  Central  to  C^ISR  operations  is 
how  it  contributes  to  the  development  of  an 
accurate  picture  of  battle  truth,  JWARS 
represents  such  C^ISR  information  flows 
in  the  synthetic  perceptions  of  battle  truth 
that  are  used  in  the  command  and  control 
of  all  operational  decisions.  And,  as  in  live 
combat,  determining  the  difference 
between  the  real  versus  perceived  truth  of 
the  battlefield  is  central  to  determining  suc¬ 
cess  on  the  battlefield.  How  a  staff  or  pilot 


another  local  attraction  is  the  gated  com¬ 
munity  of  Pebble  Beach.  Popular  destina¬ 
tions  include  the  17-Mile  Drive,  with  its 
breathtaking  views  of  the  ocean,  fabulous 
houses,  Stillwater  Cove,  the  Lone  Cypress, 
and  an  array  of  world-class  golf  courses. 
Nearby,  Carmel  Valley  is  home  to  many 
wineries,  farms,  ranches  and  the  beautiful 
Garland  Ranch  Regional  Park.  Finally, 
don’t  miss  Big  Sur  with  its  Esalen  Institute, 
Henry  Miller  Memorial  Library,  Point  Sur 
Lighthouse,  camping,  hiking,  beautiful  red¬ 
wood  groves  and  excellent  whale  watch¬ 
ing. 

For  those  of  you  who  are  stiU  undecided 
about  attending  the  66th  MORSS,  we  hope 
we  have  provided  you  a  couple  of  addition¬ 
al  good  reasons  to  attend.  We  will  provide 
more  information  in  the  June  PHALANX 
to  help  reinforce  your  decision  to  request  a 
registration  packet  from  the  MORS  Office. 
Plans  are  to  include  details  for  the  tutorials, 
poster  sessions,  the  Junior/Senior  Analyst 
session,  the  Education  Session,  and  the 
Prize  Paper  Session,  as  well  as  the 
specifics  of  the  Spouse/Guest  Program. 

If  you  have  any  questions  about  the 
Symposium,  please  contact  the  MORS 
office  at  (703)  751-7290  or  CDR  Kirk 
Michealson  at  (703)  697-0064.  Make 
your  arrangements  now  to  attend  and  par¬ 
ticipate  in  the  66th  MORSS  at  NPS  from 
23-25  June  1998!  O 


or  tank  commander  perceives  the  enemy’s 
position  and  their  own  position  in  a  battle- 
space  relies  on  a  complex  array  of  sensor 
to  target  data  gathering  processes,  commu¬ 
nications,  history  of  similar  events,  train¬ 
ing,  etc.,  all  of  which  are  complex  dynam¬ 
ics  that  builders  of  combat  models  and 
simulations  have  been  trying  to  solve  for 
decades  with  computers  and  centuries  with 
board  games  and  other  non-automated 
means. 

The  complexity  aspect  of  JWARS  for 
this  paper  will  focus  on  the  intelligence 
fusion  process,  which  is  the  process  of 
obtaining,  sorting,  and  comparing  informa¬ 
tion  from  different,  disparate  sensors.  A 
key  product  from  this  intelligence  fusion 
process  is  developing  courses  of  action 
(CO A)  for  a  variety  of  threat  weapon  sys¬ 
tems  and  organizational  structures.  In 
JWARS  what  ever  the  sensors  see  on  the 
battlefield  is  what  is  used  to  make  com¬ 
mand  and  control  deductions.  JWARS 
does  not  use  ground  truth  for  decisions 


COMMENTS 

(continued  from  p.  7) 

interdisciplinary  problems  successfully. 
As  the  wise  man  said,  great  things  can  be 
accomplished  when  you  don’t  care  who 
gets  the  credit.  Whether  better  thinking 
and  actions  are  achieved  under  the  rubric 
of  OA,  IT,  systems  engineering,  manage¬ 
ment  science,  or  industrial  engineering,  the 
important  thing  is  to  help  people  think  and 
act  with  perspicacity. 

“Sg-nus” 

Notes 

1.  By  the  dictionary,  “discipline”  is  any 
branch  of  knowledge  or  teaching,  but 
academics  are  inclined  to  apply  a  more 
stringent  standard.  A  profession  is  an 
occupation  requiring  training  in  the  lib¬ 
eral  arts  or  sciences  and  advanced  study 
in  a  specialized  field. 
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needed,  but  perceived  truth.  The  degree  of 
matching  the  two  truths  is  measured  by  the 
degree  of  success  of  gathering  sufficient 
target  intelligence. 

Part  n,  which  will  address  Complexi¬ 
ty  of  Decisions  and  the  Challenges 
Ahead,  will  appear  in  the  June  PHA¬ 
LANX, 
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4th  Annual  Air  Force  Operations  Research  Symposium 

Lt.  Col.  Jeriy  Diaz  and  Dr.  Roy  Rice 


L-R:  Mr.  Frank  Campanile,  Capt.  Jeffery  Weir,  Maj.  Glenn  Rousseau,  Maj.  Robert 
Nuanes,  Mrs.  Karen  Somers,  Mr.  Allen  Murashige 


The  Fourth  Annual  Air  Force  Opera¬ 
tions  Research  Symposium  (AFORS) 
was  held  at  the  Air  Force  Institute  of 
Technology  (AFTT),  Wright-Patterson  AFB, 
OH,  22-23  October  1997.  The  symposium 
featured  professional  exchanges,  briefings  on 
military  and  civilian  career  management, 
updates  on  current  AF  policies  on  analysis, 
modeling  and  simulation,  and  MAJCOM 
“job  fairs.”  Hosted  by  the  AFTT  OR  Depart¬ 
ment,  the  symposium  was  sponsored  by  the 
AFMC  Office  of  Aerospace  Studies  (OAS). 
Briefings  from  the  conference  can  be  found 
at  www.plk.af.mil/oas. 

This  year’s  theme,  “Air  Force  Analysis: 
Proud  of  the  Past  —  Poised  for  the  Future!” 
was  highlighted  by  keynote  speaker  Mr. 
Clayton  Thomas,  Chief  Scientist  of  the  Air 
Force  Studies  and  Analyses  Agency.  His 
presentation  on  “Fifty  Years  of  Air  Force 
Analysis”  was  of  particular  significance  with 
1997  being  the  50th  Anniversary  of  the  Air 
Force. 

At  AFORS,  Brig  Gen  James  “Sandy” 
Sandstrom,  Director  of  Command  &  Con¬ 
trol,  HQ  USAF/XOC,  recognized  the  Ana¬ 
lysts  of  the  Year.  These  awards  honor  the 
top  AF  military  and  civilian  analysts  for  their 
outstanding  contribution  to  unit  mission, 
impact  on  an  AF  decision,  and  improvements 
in  OR  methodology.  The  four  award  cate¬ 
gories  are:  1)  Company  Grade  Analysts  of 
the  Year;  2)  Field  Grade  Officer  Analysts  of 
the  Year;  3)  GS-7  to  GS-12  Civilian  Analysts 
of  the  Year;  and  4)  GS-13  to  GS-14  Civilian 
Analysts  of  the  Year.  This  year’s  competi¬ 
tion  was  exceptionally  keen  with  30  analysts 
nominated  from  around  the  world  and  across 
all  mission  areas.  Mr.  Allen  Murashige, 
Chief  Scientist  to  USAF/XOC,  presented  the 
awards  at  the  Analyst  of  the  Year  Luncheon. 

The  Junior  Civilian  Analyst  of  the  Year  is 
Mr.  Jeff  Beekman,  422nd  Test  and  Evalua¬ 
tion  Squadron.  Jeff  developed  a  methodolo¬ 
gy  for  approaching,  executing,  and  analyzing 
Tactics  Development  and  Evaluations 
(TD&Es),  which  he  briefed  to  the  Combat 
Air  Forces  Tactics  Review  Board.  His  rec¬ 
ommendations  were  accepted  and  used  to 
develop  the  Air  Combat  Command  test  pro¬ 
gram  for  TD&Es  involving  F-16s,  F-15Cs, 


F-15ES,  F-117S,  A-lOs  and  HH-60s.  Jeff 
also  reviewed,  improved  and  refined  the 
MLU  Test  Plan  of  European  F-16s.  As  a 
result,  Jeff  was  handpicked  by  the  F- 16  SPO 
to  represent  the  USAF  as  the  lead  analyst  for 
the  MLU  Test,  a  one  year  tour  in  the  Nether¬ 
lands. 

Mrs.  Karen  L.  Somers  is  the  Senior 
Civilian  Analyst  of  the  Year.  Karen  distin¬ 
guished  herself  while  working  for  HQ  Air 
Combat  Command  (ACC)  Plans  &  Programs 
Studies  &  Analysis  Squadron,  Langley  AFB, 
VA.  As  the  leader  of  the  “Quick  Look” 
Analysis  Team,  Karen  provided  ACC’s 
senior  leadership  with  timely  analyses 
impacting  near-term  decisions  on  critical 
issues.  During  the  past  year,  she  led  13  quick 
response  analyses  which  included  Bomber 
Force  Assessment,  C-130J  Stretch  Analysis, 
HH-60  Rescue,  B-52  Tanker  Requirement, 
Barksdale  Pharmacy  Analysis,  Combat 
Search  and  Rescue  Requirements,  and  Con¬ 
cept  of  Operations  for  Advanced  Systems  in 
Operations  Other  Than  War. 

Karen’s  analysis  played  an  important  role 
in  decisions  influencing  ACC  and  the  Com¬ 
bat  Air  Forces.  Her  C-130J  Stretch  Analysis 
was  used  by  SAF/AQ  in  preparing  Congres¬ 
sional  testimony  defending  the  Air  Force’s 
decision  not  to  procure  the  stretch  version  of 


the  C-130J.  In  addition,  the  Bomber  Force 
Assessment,  which  was  briefed  at  the  Spring 
1997  CORONA  Conference,  proved  critical 
to  revalidating  the  need  for  the  B-1  as  the 
“backbone”  of  the  bomber  fleet. 

The  Company  Grade  Military  Analyst  of 
the  Year  is  Capt.  Jefifery  D.  Weir,  US  Strate¬ 
gic  Command/J5.  Jeff  was  a  key  player  in  a 
highly  select  group  performing  START  III 
analysis  last  fall.  He  established  critical  para¬ 
meters  to  the  framework  of  this  study  that 
allowed  the  analysis  of  various  treaty  posi¬ 
tions.  His  results  were  briefed  to  the  Chair¬ 
man  of  the  Joint  Chiefs  of  Staff,  Secretary  of 
Defense,  and  the  President  of  the  United 
States.  The  President  ultimately  used  Jeffs 
analysis  to  set  the  bounds  of  negotiation  with 
Russian  President  Boris  Yeltsin.  Jeff  was 
also  a  central  figure  when  the  Secretary  of 
Defense  declined  to  fund  an  electromagnetic 
pulse  nuclear  detonation  detection.  Jeff  com¬ 
bined  OR  techniques  with  control  theory 
Kalman  filtering  to  assess  sensor  impacts. 
This  became  the  “definitive”  answer  on 
USSTRATCOM  force  management  capabili¬ 
ties  as  well  as  the  tool  to  measure  future 
alternatives.  This  analysis  initiated  a 
USSTRATCOM  J5/J6  in-depth  review  of 
battle  management  requirements  and  ulti- 
(See  AIRFORCE, p.  22) 
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(continued from  p.  21) 
mately  supported  Joint  Vision  2010  strategy. 
Finally,  Jeff  developed  a  statistical  methodol¬ 
ogy  for  annual  test  requirements  for  missile 
operational  testing  and  reporting.  His 
approach,  which  focuses  the  process  on 
warfighter  risk,  provides  realistic  testing 
rates,  and  conserves  critically  scarce 
resources  as  well  as  money. 

Because  of  their  outstanding  contribu¬ 
tions,  two  Field  Grade  Analyst  of  the  Year 
recipients  where  chosen:  Maj  Robert  A. 
Nuanes  from  HQ  Air  Force  Studies  & 
Analysis,  Washington  DC,  and  Maj  Glenn 
G.  Rousseau  from  HQ  Air  Mobility  Com¬ 
mand  Studies  &  Analysis  Flight,  Scott  AFB, 
IL.  Maj  Nuanes  distinguished  himself  by 
developing  an  innovative  methodology  that 
relates  unit  OPTEMPO/PERSTEMPO  to 
unit  stress.  As  Study  Director  for  the  CSAF 
directed  Stressed  Systems  Study,  the  AF 
impact  of  his  findings  includes:  program¬ 
ming  2  additional  Rivet  Joint  aircraft,  7  Rivet 
Joint  additional  crews,  reducing  exercises  by 
5%  in  1999  and  2000,  eliminating  QAFAs 
effective  1  January  1998,  and  establishing  a 


post-deployment  standdown  policy.  Both 
Air  Force  Magazine  and  the  AF  News  Ser¬ 
vice  published  articles  on  the  study  and  its 
positive  effects.  There  have  also  been  arti¬ 
cles  about  the  study  in  the  Washington  Post, 
Washington  Times,  The  Los  Angeles  Times, 
and  The  New  York  Times.  As  documented  in 
the  Congressional  Record,  Congressman 
Skelton  advocated  AF  action  based  on  the 
study  findings  in  a  House  floor  speech. 

Maj  Rousseau  distinguished  himself  as 
the  1997  Field  Grade  Analyst  of  the  Year  by 
leading  an  analysis  of  the  AMC  requirement 
to  comply  with  the  Global  Air  Traffic  Man¬ 
agement  (GATM)  system.  Glenn’s  analysis 
showed  that  failure  to  fund  GATM  results  in 
unacceptable  delays  delivering  troops  and 
equipment  to  theater  warfighters.  His  results 
have  been  briefed  to  the  AF  Requirements 
Oversight  Council,  the  Joint  Requirements 
Oversight  Council,  at  CORONA  TOP  to  all 
4-star  senior  leadership,  to  the  Chief  of  Staff 
of  the  AF,  and  the  Chairman  of  the  Joint 
Chiefs.  As  a  result  of  Glenn’s  analysis  the 
Deputy  Secretary  of  Defense  included 
GATM  as  an  important  initiative  in  this 
year’s  DPG,  4.5  Million  dollars  was  added  to 


A  Review  of  Military  Modeling 
for  Decision  Making,  3rd  edition 

Edited  by  Wayne  P.  Hughes,  Jr. 

Alexandria:  Military  Operations  Research  Society, 

1997,  375  pp.  $40.00  (paper) 


Intertwined  in  virtually  all  aspects  of 
military  decision-making  —  from  what 
weapons  governments  should  buy  to 
how  soldiers  should  use  them  —  is  mod¬ 
eling.  This  volume,  now  in  a  third  and 
substantially  revised  edition,  provides 
the  most  useful  overview  of  the  subject 
by  some  of  the  most  notable  figures  in 
the  field.  The  editor,  a  navy  captain  who 
has  written  authoritatively  on  a  number 
of  subjects  (naval  tactics  in  particular) 
sets  the  tone  in  a  masterly  overview  that 
stresses,  as  do  many  of  the  essays  that 
follow,  the  limitations  of  these  artificial 
and  simplified  representations  of  the 
warrior’s  world.  Driven  too  frequently 
by  underlying  assumptions  about  quanti¬ 
tative  factors  (firepower  and  numbers) 


rather  than  qualitative  realities  (morale, 
cohesion,  coordination)  military  models 
can  mislead  those  who  put  excessive 
faith  in  them  —  as  the  wildly  pessimistic 
projections  of  American  casualties  in  the 
Gulf  War  demonstrated.  A  work  that,  if 
read  with  care,  would  do  much  to  reduce 
the  simple  faith  placed  by  civilian  and 
soldier  alike  in  these  ubiquitous  attempts 
to  distill  reality  into  equations. 

This  review  can  be  found  in  Foreign 
Affairs  —  Recent  Books  on  Internation¬ 
al  Relations,  pg.  159. 

Reprinted  with  permission  of  Foreign 
Affairs  VoL  76  No.  6.  Copyright  1997 
by  the  Council  on  Foreign  Relations, 
Inc. 


AMC  FY98  budget,  and  the  House  National 
Security  Committee  recommended  a  67.7 
Million  dollars  “plus-up”  for  FY98.  Major 
Rousseau  also  developed  a  testing  strategy 
for  new  versions  of  large  simulation  models 
used  in  AMC.  His  use  of  univariate  statisti¬ 
cal  measures,  discriminant  and  cluster  analy¬ 
sis  techniques,  and  factor  analysis  to  detect 
differences  between  two  versions  of  the  same 
model  represents  a  huge  improvement  in  the 
application  of  operations  research  for  model 
development. 

In  addition  to  Analyst  of  the  Year 
Awards,  AFORS  presented  its  first  Lifetime 
Achievement  Award  to  Mr.  Frank  Cam¬ 
panile,  ASC/XRA,  whose  long  history  of 
outstanding  technical  accomplishments  has 
spanned  36  years  of  Federal  service.  His  dis¬ 
tinguished  contributions  in  aircraft  perfor¬ 
mance,  precision  navigation,  target  acquisi¬ 
tion,  sortie  generation,  electronic  warfare, 
weapons  effectiveness  and  delivery,  and  con¬ 
ventional  and  non-conventional  survivability 
and  vulnerability,  has  supported  the  highest 
level  decision  makers  in  military  aerospace 
system  development  and  acquisition.  His 
ability  to  perform  thorough  and  highly  credi¬ 
ble  analyses  has  saved  the  Air  Force  vital 
dollars  and  resulted  in  highly  capable,  afford¬ 
able  military  systems. 

Highlights  of  Mr.  Campanile’s  career 
include  the  AGILE  FALCON  STUDY 
which  investigated  the  military  utility  of  four 
different  conceptual  aircraft  configurations 
for  a  modified  F-16,  Chief  Analyst  for  the 
Dual  Role  Fighter  Evaluation,  which  sup¬ 
ported  an  Air  Force  source  selection  to 
decide  between  two  candidates,  the  F-15E 
and  the  F-16E,  for  a  new  dual  role  fighter  air¬ 
craft,  and  a  long  history  of  support  to  the  F- 
22  fighter  aircraft  program.  He  led  the  ana¬ 
lytical  activities  and  developed  the 
methodology  to  evaluate  20  concepts  and 
provided  expertise  for  cost  and  operational 
effectiveness  analyses  and  helped  refine  the 
aircraft  design. 

A  list  of  other  projects  he  has  had  a  signif¬ 
icant  role  in  reads  like  a  “Who’s  Who”  of 
combat  aircraft:  AMRAAM  Cost  Reduction 
Program,  AC-130U  Gunship  Accuracy  Test 
and  Verification,  Military  Utility  of  LAN- 
TIRN,  Advanced  Manned  Supersonic  Air¬ 
craft  (B-1),  the  Close  Air  Support  A-X  (A- 
10),  Tactical  Air-to-Air  Capability  Study  FX 
(F-15),  and  the  Tactical  Fighter  Avionics 
Study  (F-111  Mk-11  avionics).  ® 
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OPTEC  Team  Wins  Top  ORSA  Award 

Cindy  McGovern 
OPTEC  Public  Affairs 


L-R:  Koon  Kityu,  John  Cox,  Thomas  Zeberlein,  Lt  Col  William  Carlton,  Frank  Pipicel- 
la.  Col  Richard  Sayre,  Maj.  Gen.  Larry  G.  Lehowicz  ( OPTEC  Cmdr) 


The  Operational  Test  and  Evaluation 
Command’s  (OPTEC)  Task  Force 
XXI  Advanced  Warfighting  Exper- 
imentation  Team  received  the  Dr.  Wilbur 
B.  Payne  Memorial  Award  for  Excellence 
in  Analysis  for  best  group  analysis  for 
1997.  The  Payne  Award  is  presented  by 
the  Deputy  Under  Secretary  of  the  Army 
(Operations  Research)  (DUSA  (OR))  and 
recognizes  technical  achievement  in  Army 
operations  research  and  systems  analysis. 

One  group  award  and  one  individual 
award  are  presented  annually.  The  group 
award  is  intended  to  recognize  a  team  that 
provides  an  exceptional  product  and  is  not 
intended  to  recognize  an  organizational 
entity  for  the  totality  of  its  work  output.  In 
the  case  of  the  OPTEC  team,  the  product 
cited  for  the  award  is  the  ‘Task  Force  XXI 
Advanced  Warfighting  Experiment  (AWE) 
Live  Experiment  Assessment  Report.”  In  a 
letter  announcing  the  award,  Mr.  Walter 
Hollis,  DUSA  (OR)  stated,  “This  report 
reflects  timely  analysis  of  experimental 
concepts  and  systems  which  are  of  utmost 
importance  to  the  21st  century  Army.” 

OPTEC  was  tasked  by  the  Chief  of  Staff 
of  the  Army  with  providing  an  unbiased 
view  of  the  AWEs  to  the  Army’s  senior 
leadership.  OPTEC  had  the  lead  in  the 
Army  for  data  collection,  data  management 
and  live  assessment  of  the  Task  Force  XXI 
AWE.  OPTEC’ s  assessment  mission  was 
to  provide  independent  assessment  of  the 
Applique  and  tactical  Internet,  72  modern¬ 
ization  initiatives,  32  joint  venture  issues 
and  10  force  level  capabilities.  To  accom¬ 
plish  this  mission,  OPTEC  collected  data 
—  a  total  of  26  gigabytes  —  during  four 
rotations  at  the  National  Training  Center 
(NTC),  including  the  culminating  event,  the 
Task  Force  XXI  rotation  in  March  1997. 
The  team  also  analyzed  thousands  of  sol¬ 
dier  comments  and  conducted  interviews 
with  almost  every  commander  and  soldier 
in  the  rank  of  captain  and  above  who  par¬ 
ticipated  in  the  Task  Force  XXI  AWE. 

To  analyze  the  overwhelming  amount  of 
data,  the  OPTEC  team  focused  on  identify¬ 


ing  and  validating  trends  rather  than  the 
cause  and  effect  of  specific  events.  The 
OPTEC  team  also  used  a  three-tiered 
analysis  strategy  that  focused  first  on  mili¬ 
tary  subject  matter  experts  and 
observer/controller  comments.  The  second 
tier  focused  on  the  analysis  of  available 
radio  data,  other  electronic  data  and  battle 
outcomes  and  served  as  verification  for  the 
personal  observations.  The  final  tier 
focused  on  modeling  and  simulation  data 
which  was  provided  to  TRADOC’s  Analy¬ 
sis  Center  at  White  Sands  Missile  Range  to 
support  their  constructive  and  virtual  analy¬ 
sis  efforts. 

In  addition  to  a  comprehensive  final 
assessment  report,  the  OPTEC  team  pro¬ 
vided  four  interim  assessment  reports  and 
nightly  emerging  insights  briefings  at  NTC 
to  senior  leaders  across  the  country  via 
video  teleconferencing.  OPTEC’ s  Task 
Force  XXI  reports  have  provided  input  for 
Army  funding  decisions  and  the  Warfighter 
Rapid  Acquisition  Program  (WRAP), 
development  of  system  requirements  and 


information  architectures,  and  organization¬ 
al  changes  for  the  Army’s  21st  century  dig¬ 
itized  division  and  corps. 

The  OPTEC  report  was  truly  a  team 
effort  with  14  members  from  three  different 
OPTEC  elements,  the  Operational  Evalua¬ 
tion  Command  (OEC)  located  in  Alexan¬ 
dria,  Virginia,  the  Evaluation  Analysis 
Center  (EAC),  located  at  Aberdeen  Proving 
Ground,  Maryland,  and  the  Test  and  Exper¬ 
imentation  Command  (TEXCOM)  located 
at  Fort  Hood,  Texas. 

The  Payne  Award  was  initiated  in  1980 
as  the  Department  of  the  Army  Systems 
Analysis  Award.  In  1987,  the  award  was 
renamed  in  honor  of  Dr.  Wilbur  B.  Payne, 
the  first  DUSA  (OR).  Payne  is  also 
remembered  for  his  mentorship  of  the 
Army’s  best  and  brightest  analysts. 

The  winner  of  the  individual  award  is 
MAG  Patrick  J.  DuBois,  Ph.D.,  from  the 
US  Army  Analysis  Concepts  Agency. 
Please  see  the  June  PHALANX  for  a  full 
article  on  MAJ  DuBois  O 
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An  example  of  this  level  of  testing  is 
integration  of  tank,  infantry  fighting  vehi¬ 
cles,  and  helicopters  in  a  tactical  training 
simulator.  The  type  of  experiment  will 
most  likely  involve  the  higher  end  and 
more  difficult  scenarios  in  the  simulator’s 
training  matrix.  Testing  that  stresses  the 
system  is  the  focus  of  this  level. 

The  final  organizational  level  for  test¬ 
ing  is  acceptance  testing.  The  benefit  of 
interim  experimentation  is  reduced 
requirements  at  the  final  testing  level. 
The  testing  at  this  level  should  focus  on 
critical  verification  and  validation  issues 
that  have  either  been  carried  over  from  a 
previous  level  with  errors,  or  have  special 
interest  from  the  producer  or  user.  The 
team  leaders  and  testing  team  chief  should 
jointly  decide  which  issues  need  to  be 
tested  again.  Because  lEDM  can  be  inte¬ 
grated  into  existing  acquisition  systems, 
the  Milestone  3  production  tests  required 
in  the  US  Army  acquisition  system  is  an 
example  of  an  acceptance  level  test. 

The  total  number  of  interim  experi¬ 
ments  required  is  the  total  of  each  of  the 
four  levels.  This  number,  as  well  as  the 
scope  and  required  participants,  should  be 
outlined  and  agreed  upon  formally  in  a 
master  test  plan.  This  plan  should  be  a 
fluid  document  because  system  and 
acceptance  level  testing  requirements  can 
be  affected  by  previous  interim  experi¬ 
ment  performance.  The  last  check  in  the 
first  step  of  the  DEDM  model  is  ensuring 
that  all  written  system  specifications  and 
requirements  have  been  captured  in  at 
least  one  of  the  designated  levels. 

In  this  article  the  current  efforts  of  veri¬ 
fication  and  validation,  and  problems 
associated  with  them,  were  discussed.  A 
process  for  approaching  verification  and 
validation  of  simulators  has  been  suggest¬ 
ed.  A  need  has  been  identified  for  a  struc¬ 
tured  approach  to  verification  and  valida¬ 
tion  through  interim  experiments.  The 
subsequent  article  in  this  series,  Interim 
Experiment  Design  Model  -  A  Verifica¬ 
tion  and  Validation  Process  for  Training 
Simulators,  will  provide  an  approach. 

Statement  of  Contribution 

As  technology  increases  and  costs  for 
training  the  new  technology  escalate,  sim¬ 
ulators  will  increase  in  importance  and 
complexity.  This  will  create  a  more 


urgent  need  for  a  structured  approach  to 
verification  and  validation  of  new  simula¬ 
tors.  The  approach  needs  to  minimize 
effort/cost  and  to  provide  a  maximum  of 
information.  The  first  article  identifies  the 
state  of  current  verification  and  validation 
efforts  and  focuses  on  the  development  of 
a  structured  approach  for  verification  and 
validation  of  training  simulators.  The 
approach  combines  in  a  unique  way  prin¬ 
ciples  gathered  from  current  verification 
and  validation  theory,  systems  engineer¬ 
ing  principles,  quality  assurance  theory 
and  experimental  design  techniques.  The 
interim  experiment  design  model 
described  in  article  two  is  a  management 
tool  that  will  improve  the  organization  of 
interim  experimentation  and  verification 
and  validation  efforts.  This  improved 
organization  allows  for  detection  of  errors 
early  in  the  product  life  cycle.  The  effi¬ 
ciency  of  subsequent  experiments  will  be 
enhanced  as  a  result.  The  early  detection 
of  errors  coupled  with  the  improved  effi¬ 
ciency  will  provide  verification  and  vali¬ 
dation  at  reduced  cost. 

Abstract 

This  series  of  articles  centers  on  the  use 
of  interim  experimentation  for  verification 
and  validation  of  training  simulators.  The 
first  article  identifies  the  state  of  current 
verification  and  validation  efforts  and 
focuses  on  the  development  of  a  struc¬ 
tured  approach  for  verification  and  valida¬ 
tion  of  training  simulators.  The  second 
article  outlines  a  methodology  using  inter¬ 
im  experimentation  for  verification  and 
validation.  The  result  is  an  Interim  Experi¬ 
ment  Design  Model. 

The  issue  of  verification  and  validation 
is  a  primary  concern  for  today’s  opera¬ 
tions  research  analyst.  Applying  verifica¬ 
tion  and  validation  to  training  simulators 
results  in  a  combination  of  principles 
gathered  from  current  verification  and  val¬ 
idation  theory  of  simulation  models,  sys¬ 
tems  engineering  principles,  quality  assur¬ 
ance  theory,  and  experimental  design 
techniques. 

The  interim  experiment  design  model 
provides  a  management  tool  that  will 
improve  the  organization  of  interim  exper¬ 
imentation  and  verification  and  validation 
efforts.  This  improved  organization 
allows  for  detection  of  errors  early  in  the 
product  life  cycle.  The  efficiency  of  sub¬ 
sequent  experiments  will  be  enhanced  as  a 


result.  The  early  detection  of  errors  cou¬ 
pled  with  the  improved  efficiency  will  pro¬ 
vide  verification  and  validation  at  reduced 
cost. 

Pace,  D.K.,  (1995).  A  Modest  V&V  Pro¬ 
posal.  PHALANX  (28:4),  16-17. 
Gledhill,  D.W.,  (1994).  The  Use  of  Case 
Tools  as  an  Aid  to  the  Verification  of 
Complex  Software  Systems,  Simula¬ 
tion,  63:5,  329-336. 

Dunham,  J.R.,  (1989,  May).  V&V  In  The 
Next  Decade.  IEEE  Software,  47-53, 
Headquarters,  Department  of  the  Army, 
(1993).  Verification,  Validation,  and 
Accreditation  of  Army  Models  and  Simu¬ 
lations  (Department  of  the  Army  Pam¬ 
phlet  5-11).  Washington,  DC. 
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Artificial  Intelligence 
(AI)  and  Simulation 
Workshop 

The  fifteenth  National  Conference 
on  Artifical  Intelligence  (AAAI-98) 
win  hold  an  AI  and  Simulation  Work¬ 
shop  as  part  the  conference  program. 
The  workshop  will  be  held  during  26 
-31  July,  1998  meeting  in  Madison, 
Wisconsin.  For  further  information 
contact  Dick  Modjeski  (Chair)  at 
worldnet.att.net  or  Tom  Shook  at 
tshook@  mindspring.com. 


PHALANX 


24 


March  1998 


COMBAT  ANALYSIS 


What  Every  G^od  OR  Analyst  Should  Know 
About  Bayesian  Networks! 


h,:  has  capability 
hy  no  capability 


e,:  supports  h| 
Cj:  supports  hj 


p(E=eJH=hi)  =  q  =.6 

p(E=e2|H=hO  =  (l-q)  =.4 

p(E=ei|H=h2)  =  r  =.l 
p(E=e2|H=h2)  =  (l-r)  -.9 


Conditional 

Probability 

Distributions 


p(H=h,)  =  p  =  .5 

p(E=e,)  =  (p*q)+[(l-p)*r)]  =.35 

p(H=h2)  =  (l-p)  =  .5 

p(E=e2)  =  [p*(l-q)]+[(l-p)*(l-r)]  =  .65 

1  Prior  Marginal  Probability  Distributions 

p(e,  ,h,)  =  .175 

P(®2  ’^l)  ~ 

p(e,  ,h2)  =  .175 

p(e2  ,h2)  =  .325 

State  Probabilities 

Figure  1.  Fully  Specified  Bayes  Network  (Prior) 


Two  things  are 
almost  always 
true  when  a 
military  leader  or  a 
civilian  policy  maker 
asks  military  opera¬ 
tions  researchers  to 
assist  in  addressing 
issues  and  decision 
problems.  First,  the 
leader  cares  about  the 
problem  for  more  than 
one  reason.  Normally, 
these  reasons  are  con¬ 
nected  with  a  highly 
complex  set  of  relationships.  If  thinking  the 
problem  through  was  easy  he  or  she  would 
just  complete  an  intuitive  analysis  and  make 
an  appropriate  decision.  Second,  the  future 
can  not  be  predicted  with  certainty.  Uncer¬ 
tainty  is  a  fact  of  life,  particularly  in  the  area 
of  military  operations  research. 

The  Bayesian  network  is  an  emerging 
modeling  technique  that  provides  OR  ana¬ 
lysts  and  computer  scientists  a  new  and 
extremely  powerful  mechanism  for  integrat¬ 
ing  necessarily  informal  model  construction 
techniques  with  the  formalism  required  for 
computationally  generated  inference  under 
conditions  of  uncertainty.  Bayes  Nets  are 
currently  being  applied  successfully  to 
many  important  analytical  problems, 
including  some  military  problems.  Addi¬ 
tionally,  Bayesian  networks  are  often  found 
embedded  into  larger  scale  decision  support 
systems  and  artificial  intelligence  applica¬ 
tions  (see  Table  1).  Readily  available  com¬ 
mercial  software  packages  have  capitalized 
on  recent  advances  in  graphical  displays, 
computational  algorithms,  and  computer 


processing  speed.  These  packages  support 
rapid  construction  of  complex  Bayes  net¬ 
work  models.  This  has  led  to  rapid  increas¬ 
es  in  the  variety  and  practicality  of  such 
models.  Soon,  they  will  become  another 
standard  tool  of  military  operations  analysis. 

This  introduction  to  Bayesian  networks 
briefly  answers  the  following  questions: 
What  is  a  Bayesian  network?  What  are 
they  good  for?  How  can  I  find  out  more 
about  them? 

What  are  Bayesian  Networks? 

A  Bayesian  network  is  a  set  of  interde¬ 
pendent  Bayesian  inferences.  That  is,  some 
inferences  in  the  model  depend  on  knowl¬ 


edge  that  is  encoded  elsewhere  in  the 
model.  The  relationships  in  these  models 
are  often  complex.  To  deal  with  these  com¬ 
plexities  Bayesian  networks  are  normally 
formulated  using  graphical  techniques  that 
make  it  simultaneously  understandable  to  a 
human  formulating  the  model  (like  a 
domain  expert)  and  to  a  computer  for  math¬ 
ematical  computation.  Some  researchers  in 
the  field  refer  to  Bayes  nets  as  knowledge 
maps^  belief  networks,  or  causal 
networks.^ 

The  computational  foundation  for  Bayes 
nets  is  Bayes  Theorem.  That  theorem  states 
that: 

Pr  (Hi\  £;•)  oc  2  Pr  (s1  Hj)  Pr  (Hi). 

In  other  words,  it  says  that  the  probabili¬ 
ty  (belief)  associated  with  a  hypothesis  (Z/^) 
may  change  as  evidence  (Ej)  is  collected. 
That  change  is  proportional  to  the  sum  of 
the  probabilities  that  one  will  observe  the 
evidence  when  the  hypothesis  is  tme.  Fig¬ 
ure  1  is  an  example  of  a  fully  specified, 
albeit  very  simple,  Bayes  net.  The  nodes 
(5^^  NETWORKS,/?.  26) 


Table  1:  Sample  Uses  of  Bayesian  Networks 


Used  by 

Purpose 

Microsoft 

Windows  95  online  troubleshooting 

Selected  intelligence  agencies 

Sensor  fusion 

Medical  laboratories 

Disease  diagnosis  from  test  results 

Oil  companies 

Oil  price  forecasting 

Lieutenant 
Colonel  Dan 
Maxwell 
JWARS 
OSD(PA&E) 
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Figure  2.  Posterior  Bayes  Network 


Figure  3.  A  more  complex  Bayes  net 


NETWORKS 

(continued from  p,  25) 

represent  variables  in  the  model  and  the 
arcs  identify  relationships  that  may  exist 
among  those  variables.  The  relationships 
are  expressed  in  the  model  as  conditional 
probability  distributions  or  possibly  deter¬ 
ministic  functions.  Information  concerning 
these  relationships  is  often  collected  from 
experts.  However,  it  could  also  be  collect¬ 
ed  through  physical  tests  and  experimenta¬ 
tion,  historical  data,  or  as  an  output  from  a 
simulation  model.  This  particular  network 
represents  a  joint  probability  distribution 
involving  the  variables  H  and  E.  It  has  four 
states.  This  is  referred  to  as  a  prior  distrib¬ 
ution  because  it  identifies  the  level  of  belief 
associated  with  the  model  variables  before 
evidence  is  collected. 

Imagine  a  hypothesis  (H)  concerning  a 
potential  aggressor  nation’s  possible  pos¬ 
session  of  nuclear  weapons.  We  begin  with 
the  likelihood  of  the  possession  of  these 
weapons  as  a  “toss  up.”  This  is  reflected 
by  assigning  a  probability  of  .5  to  each  pos¬ 
sibility.  Our  nation  collects  evidence  (E) 
from  various  sources  that  affect  the  belief 
that  a  nation  possesses  these  weapons  of 
mass  destruction.  The  meaning  of  this  evi¬ 
dence  is  often  uncertain.  This  is  why  con¬ 
ditional  probability  distributions  are  used. 
For  example:  If  we  observe  ej  it  increases 
the  belief  that  a  nuclear  capability  exists. 
This  is  identified  as  a  probability  of  .6. 
However,  there  is  a  40  percent  chance  that 
e|  could  be  observed  even  if  there  was  no 
nuclear  capability.  Conversely,  ^2  could  be 
observed  if  a  nuclear  capability  exists,  but 
only  ten  percent  of  the  time.  Ninety  per¬ 
cent  of  the  time  e2  would  correctiy  indicate 
that  the  nation  did  not  possess  a  nuclear 
capability. 

Unfortunately  we  cannot  directly 
observe  the  hypothesis.  We  observe  evi¬ 
dence.  In  this  case  pictures,  intelligence 
reports,  and  other  sources  might  serve  as 
relevant  evidence.  The  information  in  Fig¬ 
ure  1,  plus  an  observation  of  the  evidence, 
and  Bayes  theorem  provide  the  three  keys 
to  rationally  adjust  our  belief  in  H.  Figure 
2  demonstrates  how  this  works.  Once  ej  is 
observed  to  be  true,  application  of  Bayes 
rule  increases  the  belief  associated  with  hj 
significantly.  These  probabilities  associated 
with  H  are  now  called  “posterior”  distribu¬ 
tions. 

This  example  is  So  elementary  that  it 
could  easily  be  solved  without  elaborate 


computational  assistance.  There  is  no 
requirement  to  hierarchically  apply  Bayes 
theorem  to  interdependent  inferences  or  to 
integrate  multiple  sources  of  information. 
There  are  few  real-world  situations,  how¬ 
ever,  that  are  so  obvious.  Figure  3  is  an 
example  of  a  slightly  more  complex  net¬ 
work  that  illustrates  what  can  happen  when 
interdependent  inferences  and  multiple 
information  sources  are  introduced.  This 
model  has  only  seven  nodes  and  eight  arcs 
(It  is  also  a  simple  prototype).  If  one 
assumes  that  each  node  has  two  states, 
there  could  be  127  (2^-1)  probabilities  that 


would  require  elicitation.  Fortunately,  the 
structure  of  the  problem,  as  expressed  by 
the  eight  arcs  in  the  graph  reduces  the  elici¬ 
tation  requirement  to  18  probability  judg¬ 
ments.  In  general  this  elicitation  savings 
continues  to  grow  as  the  networks  get  larg¬ 
er. 

There  are  two  types  of  applications  for 
which  Bayes  nets  are  particularly  well  suit¬ 
ed,  The  first  type  of  application  is  diagnos¬ 
tic.  What  should  I  believe  about  the  world, 
given  the  evidence  I  have  collected?  The 
question  concerning  ownership  of  nuclear 
weapons  is  of  this  type.  The  second  appli- 
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A  list  of  references  that  address  Bayesian  networks,  with  the  audience  it  would  appeal  to 
is: 


Source 

Audience 

Decision  Analytic  Networks  in  Artificial 
Intelligence,  Matzkevich  and  Abramson 

Introductory.  A  must  for  programmers. 

(Has  a  great  reference  list.) 

Influence  Diagrams,  Belief  Nets,  and 

Decision  Analysis,  Oliver  and  Smith 

Intermediate  to  advanced.  For  analysts 
who  will  be  constructing  models. 

Probabilistic  Reasoning  in  Intelligent 

Systems,  Pearl 

Intermediate  to  advanced.  For  analysts 
who  will  be  constructing  models. 

Probabilistic  Reasoning  in  Expert  Systems: 
Theory  and  Algorithms,  Neapolitan 

Advanced.  For  those  who  want  to  know 
what  makes  something  tick. 

Supporting  Decision  Makers  in  Future 
Conflicts:  A  Decision  Theoretic 

Perspective,  Maxwell 

Introductory.  For  the  decision  maker  or 
analyst  who  is  unfamiliar  with  Bayes  nets, 
(includes  a  discussion  of  influence  diagrams) 

Influence  Diagrams,  Howard  &  Matheson 

Introductory  to  advanced. 

This  is  the  seminal  research  article. 

Bayesian  Networks  Without  Tears,  Charniak 

Introductory,  Mandatory  reading 

cation  is  for  predictive  modeling.  Given  a 
set  of  events  (evidence),  what  is  the  likeli¬ 
hood  that  something  will  happen?  The 
sample  network  in  Figure  3  is  an  example 
of  this  type.  In  both  instances,  information 
developed  through  exploratory  modeling  on 
Bayes  nets  is  easily  extended  into  Decision 
Diagrams  that  can  be  modeled  using  other 
decision  analytic  techniques  and  software. 

Bayes  nets  are  not,  however,  a  modeling 
panacea.  In  some  situations  analysts  can 
rather  easily  “outsmart”  themselves.  First, 
Bayes  nets  (and  Bayes  net  software)  are  not 
usually  the  best  vehicle  for  collecting  expert 
input  regarding  uncertainty.  There  are  other 
techniques  and  commercially  available  soft¬ 
ware  for  accomplishing  that  task.  Second, 
in  conditions  where  the  uncertainty  sur¬ 
rounding  the  uncertainty  is  high,  Bayes  nets 
can  give  the  “illusion  of  accuracy”  by  using 
a  computationally  complex  model.  In  such 
cases  analysts  may  want  to  resort  to  simpler 
methods.  Finally,  situations  that  have  a 
very  significant  temporal  aspect  can  be  dif¬ 
ficult  to  capture  in  a  Bayes  net:  the  joint 
probability  distribution  that  is  formed  is  sta¬ 
tic.  Therefore,  temporal  relationships  must 
be  provided  as  part  of  the  model  formula¬ 
tion  process.  This  very  often  results  in 
computationally  infeasible  models  because 
of  the  complexity  of  these  relationships. 

There  are  a  myriad  of  subtle,  but  very 
important,  issues  that  must  be  considered  in 
the  development  of  all  inferential  models. 
The  challenges  to  building  good  Bayesian 
networks  identified  here  are  also  challenges 
to  varying  degrees  for  all  other  inferential 
modeling  techniques.  Interested  readers 
should  refer  to  research  in  the  areas  of  evi¬ 
dence,  intelligence  modeling,  and  descrip¬ 
tive  decision  theory.^’  4 

Where  is  there  more  information? 

There  are  a  number  of  ways  to  learn 
more  about  Bayesian  networks.  One  could 
read  the  literature,  surf  the  net,  or  experi¬ 
ment  with  modeling  in  a  commercially 
available  software  package. 

There  is  also  a  continually  growing  array 
of  software  available  for  processing 
Bayesian  networks  and  influence  diagrams. 
A  review  of  many  of  these  packages  is  pub¬ 
lished  annually  in  OR/MS  Today, ^  Addi¬ 
tionally,  the  INFORMS  Decision  Analysis 
Society  maintains  a  web  site  that  provides 
links  to  the  current  hterature  and  software 
sources.  The  address  is  http://www. 
informs.org/Society/DA. 
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US  Army  Concepts  Analysis  Agency  and 
provided  analysis  in  support  of  many  DoD 
studies.  He  holds  a  Ph.D.  in  Information 
Technology  from  George  Mason  University, 
an  MBA  from  Long  Island  University,  and  a 
B.S.  in  Criminal  Justice  from  Rochester 
Institute  of  Technology.  O 
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CAA  to  Celebrate  Silver  Anniversary;  All  Alumni  Invited 

E.  B.  Vandiver  III 

Director,  US  Army  Concepts  Analysis  Agency 


On  15  January  1998  the  US  Army 
Concepts  Analysis  Agency 
(CAA)  completed  its  first  twenty- 
five  years  of  providing  responsive  analysis 
support  to  Headquarters,  Department  of  the 
Army  and  other  Army  decision  makers. 
The  CAA  Silver  Anniversary  will  be  cele¬ 
brated  at  the  annual  Anniversary  Dinner 
Dance  on  25  April  1998  at  the  Fort  Myer, 
Virginia,  Officers  Club. 

CAA  was  created  in  the  1973  post- 
Vietnam  reorganization  of  the  Army  that 
went  under  the  name  Project  STEAD¬ 
FAST.  In  this  reorganization  the  previous 
Continental  Army  Command  (CONARC) 
and  the  Combat  Development  Command 
(CDC)  were  realigned.  Combat  develop¬ 
ments  was  strengthened  by  combining  it 
with  training  in  the  Training  and  Doctrine 
Command.  The  span  of  attention  of 
CONARC  was  reduced  by  creating  a  new 
Forces  Command  that  was  to  focus  solely 
on  the  readiness  of  Army  Forces.  Thus 
from  CONARC  and  CDC  the  Training  and 
Doctrine  Command  (TRADOC)  and  the 
Forces  Command  (FORSCOM)  emerged. 

STEADFAST  also  centralized  opera¬ 
tional  testing  and  force  analysis  through 
the  creation  of  two  new  Headquarters, 
Department  of  the  Army  agencies:  the 
Operational  Test  and  Evaluation  Agency 
(OTEA)  for  operational  testing  and  the 
Concepts  Analysis  Agency  (CAA)  for 
force  analysis. 


The  driving  intellect  behind  STEAD¬ 
FAST  was  the  late  General  William  E. 
DePuy  who  at  that  time  was  Assistant 
Vice  Chief  of  Staff  of  the  Army.  That  the 
organizational  realignments  of  STEAD¬ 
FAST  have  now  endured  for  twenty-five 
years  is  a  tribute  to  the  incisive  insight  and 
far  reaching  foresight  of  General  DePuy. 

Much  has  changed  at  CAA  over  twen¬ 
ty-five  years  but  essential  core  values  have 
endured.  CAA  had  a  peak  strength  of 
about  325:  it  is  now  authorized  178.  CAA 
once  did  mostly  studies  of  some  length  (9 
months  to  1  1/2  years)  with  manpower 
loading  counted  in  man-years.  It  now  does 


preponderantly  quick  reaction  analyses  that 
take  days,  weeks,  or  a  few  months  and 
commensurate  manpower.  CAA  once 
focused  almost  exclusively  on  analysis  of 
the  Central  Front  in  Germany:  it  now  con¬ 
ducts  analysis  ranging  over  all  areas  of  the 
globe.  CAA  once  almost  exclusively  sup¬ 
ported  the  Office  of  the  Deputy  Chief  of 
Staff  for  Operations  and  Plans:  today 
CAA  supports  a  broad  spectrum  of  Army 
decision  makers  in  both  Headquarters, 
Department  of  the  Army  and  Army  Major 
Commands.  CAA  once  had  a  single  giant 
computer,  and  now  has  a  dedicated  com- 
(See  CAA,/7.  29) 


Lessons  Learned  and  Future  Directions 
Mini-Symposium  and  Workshop, 

7-9  April  1998 


The  legislation  that  mandated  the 
1997  Quadrennial  Defense  Review 
(QDR)  calls  for  another  QDR  in 
2001.  The  military  analytic  community 
can  improve  the  quality  of  the  analysis 
supporting  the  next  QDR  by  drawing  upon 
the  lessons  of  the  last  one. 

With  this  in  mind,  MORS  has  orga¬ 
nized  a  Mini-Symposium  and  Workshop  to 
address  the  question  of  “QDR  Analysis: 
Lessons  Learned  and  Future  Directions.” 
It  will  be  held  at  The  Johns  Hopkins  Uni¬ 
versity  Applied  Physics  Lab  in  Laurel, 
Maryland 

The  Mini-Symposium  on  April  7^  will 
consist  of  presentations  by  high-level  deci¬ 
sion  makers  from  OSD,  the  Joint  Staff,  the 
Services,  and  Congress  —  as  well  as  a 
member  of  the  National  Defense  Panel  and 
several  experienced  outside  observers. 
These  presentations  will  reflect  upon  the 
lessons  that  can  be  drawn  from  the  1997 
QDR  process  and  will  also  look  to  the 
future. 


The  Workshop  on  8-9  April  1998  will 
consist  of  working  groups  to  explore  ways 
of  improving  the  analysis  that  underlay  the 
major  topics  of  the  QDR.  There  will  also 
be  presentations  on  alternative  ways  of 
thinking  about  the  problems  addressed  by 
the  QDR. 

Participants  may  attend  just  the  sympo¬ 
sium  on  7  April.  All  participants  in  the  8-9 
April  workshop  should  also  attend  the  7 
April  symposium. 

MORS,  as  the  professional  association 
of  military  analysts,  hopes  that  this  gather¬ 
ing  will  identify  actions  to  be  taken  and 
research  to  be  carried  out  over  the  next  30 
months  that  will  improve  both  the  intellec¬ 
tual  quality  and  the  value  of  the  next  QDR. 

For  further  information  on  the  Mini- 
Symposium/Workshop,  checks  the  MORS 
web  page  at  www.mors.org.  Or  you  can  e- 
mail  (morsoffice@aol.com)  or  call  (703- 
751-7290)  the  MORS  office  to  request  an 
application  packet.  O 
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CAA 

(continued from  p.  28) 

puter  (or  two)  on  every  analyst’s  desk  net¬ 
worked  throughout  CAA,  the  Army  and 
the  defense  community.  The  one  thing  that 
has  not  changed  is  CAA’s  unwavering 
dedication  to  providing  the  Army’s  top 
leadership  with  responsive,  high  quality 
analysis  support. 

To  our  Silver  Anniversary  Celebration 
we  are  inviting  the  STEADFAST  person¬ 
nel  involved  with  the  creation  of  CAA,  the 
DA  Staff  personnel  who  created  the  origi¬ 
nal  plans  for  CAA,  the  CAA  Implementa¬ 
tion  Planning  Group,  and  all  the  previous 
senior  leadership  of  CAA.  All  CAA 
Alumni  are  invited  to  come  join  in  our  cel¬ 
ebration.  For  information  and  reservations 
contact  Ms.  Renee  Carlucci  at  (301)  295- 
5270  (e-mail:  carlucci@caa.army.mil);  or 
Mr.  Neal  Siegel  at  (301)  295-5255  (e-mail: 
siegeln@caa.army.mil),  or  FAX  (301) 
295-3874.  O 


SIMTECH  2007  - 
Session  2 

The  Organizing  Committee  for  Sim¬ 
ulation  Technology  (SIMTECH) 
2007  has  decided  to  postpone  the 
second  Workshop,  which  had  been  sched¬ 
uled  for  5  -  7  March  1998.  There  are  sever¬ 
al  reasons  for  the  postponment: 

•  It  has  taken  longer  than  anticipated  to 
document  fully  the  deliberations  of  the 
first  workshop  (which  was  conducted  16 
-  18  December  1997).  It  was  deemed 
important  to  capture  and  convey  the 
results  of  that  meeting  to  estabhsh  a  firm 
foundation  for  the  second  Workshop. 

•  It  became  apparent  that  the  previously 
scheduled  dates  for  the  second  Work¬ 
shop  were  inconvenient  for  a  number  of 
key  participants  (e.g.,  academics;  indi¬ 
viduals  involved  in  other  M&S  commu¬ 
nity  activities). 

Consequently,  the  second  Workshop 
has  been  tentatively  rescheduled  for  1 1  - 
13  August  1998.  The  location  for  that 
Workshop  is  to  be  announced.  O 


The  Military  Operations  Research  Journal 
Call  for  Papers 

Special  Issue  of  Military  Operations  Research  on 

Warfare  Analysis  and  the  New  Sciences 


In  September  1997  MORS  held  a 
Mini-symposiumAVorkshop  on  Warfare 
Analysis  and  Complexity.  The  Mini¬ 
symposium  drew  165  participants  who 
were  enthusiastic  about  the  use  of  New 
Sciences  in  mihtary  operations  research. 
The  recent  article  in  PHALANX  summa¬ 
rized  much  of  the  discussion  regarding 
the  apphcabihty  of  the  New  Sciences  to 
problems  in  military  operations 
research.  We  want  to  expand  the  discus¬ 
sion  and  consider  technical  apphcations 
of  the  New  Sciences  to  specific  prob¬ 
lems  in  operations  analysis.  The  special 
issue  of  Military  Operations  Research 
will  be  focused  on  this  topic.  The  pur¬ 
pose  of  this  special  issue  is  to  assess, 
clarify,  and  discuss  the  effectiveness  of 
novel  New  Sciences  techniques  and  to 
propose  improvements  to  mihtary  oper¬ 
ations  analysis. 

The  New  Sciences  are  meant  to 
include  chaos  and  complexity  theory, 
edge  of  chaos,  complex  adaptive  sys¬ 
tems,  and  more  generally,  any  tech¬ 
niques  for  rapidly  scanning  the  solution 
space  of  a  computationally  complex 


problem  in  order  to  determine  bound¬ 
aries  of  behavior  and  regions  where 
rapid  changes  occur.  All  papers  will  be 
unclassified.  In  accordance  with  MOR 
editorial  pohcy,  we  require  certification 
from  a  senior  decision-maker  when 
applicable. 

Interested  authors  should  submit 
abstracts  by  31  May  1998  and  papers 
by  30  September  1998.  The  papers 
should  be  submitted  in  accordance  with 
our  current  editorial  pohcy.  Ah  papers 
will  be  refereed. 

Please  contact  me  if  you  are  interest¬ 
ed  in  authoring  a  paper  for  this  special 
issue. 

DR.  JULIAN  PALMORE 

Guest  Editor,  Military  Operations 

Research 

University  of  Dhnois  at  Urbana-Cham- 
paign 

1409  W.  Green  Street 

UrbanalL  61801-2975 

Phone:  217-333-0407  (with  voice  mail) 

Email:  palmore@uiuc.edu 


Announcing  MORS-TRAC  Open  House: 
TRADOC  Analysis  Center  (TRAC)  -  Monterey 


Need  a  break  from  the  66th  MORSS? 
Come  visit  TRAC-Monterey’s  open  house! 
Ah  MORSS  attendees  are  invited  to  see  cut¬ 
ting  edge  computer  simulation  demonstra¬ 
tions,  learn  about  TRAC  projects,  and  learn 
about  the  history  of  TRAC-Monterey.  The 
open  house  includes  daily  demonstrations  of 
constructive  combat  simulations  in  stand 
alone  and  distributed  modes.  Hands-on 
opportunities  to  build  and  fight  battlefield 


scenarios  using  the  Janus  high-resolution 
simulation.  Fly  the  NPSNET  virtual  heli¬ 
copter  against  hostile  Janus  entities.  Don’t 
miss  this  exciting  opportunity  to  experience 
what  the  US  Army  TRADOC  Analysis  Cen¬ 
ter  is  all  about!  It  wiU  be  held  daily  23-25 
June  from  1200-1330  (refreshments  will  be 
provided).  For  Details  contact  SFC  Chris 
Augustine  at  (408)  656-3086/4059  or  by  e- 
mail  at  augustic@mtry.trac.  nps.navy.mil. 
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established.  The  Board  agreed.  The  new 
oificer,  the  President  Elect,  will  automati¬ 
cally  succeed  to  the  office  of  President  at 
the  conclusion  of  the  annual  symposium 
held  a  year  after  her/his  election.  Also,  the 
Board  voted  to  hold  election  for  the  new 
Executive  Council  and  new  Board  mem¬ 
bers  at  a  Board  meeting  on  Sunday  —  a 
day  earlier  than  the  past  Monday  election 
meeting.  This  earlier  election  should  pro¬ 
vide  for  committee  assignments  to  be  com¬ 
pleted  sooner  and  for  the  new  Executive 
Council  to  interact  more  with  members 
during  the  symposium.  Both  of  these  new 
initiatives  were  designed  to  improve  the 
Executive  Council  transition  and  provide 
better  focus  on  the  future  vision  of  MORS. 
Although  the  duties  of  the  President  Elect 
have  yet  to  be  defined  exactly,  it  is  antici¬ 
pated  that  the  Strategic  Planning  Commit¬ 
tee  will  report  to  the  President  Elect.  Also, 
the  President  Elect  will  participate  in  the 
Sponsors’  luncheons  —  both  at  the  annual 
symposium  and  mid-year  at  the  Pentagon. 
If  anyone  has  any  ideas  for  the  stmcture  of 


this  new  office,  please  send  them  to  the 
MORS  office.  (This  June  will  see  a  unique 
MORS  election.  Both  a  President  and 
President  Elect  will  be  elected  by  the 
Board  —  details  and  procedures  are  still 
being  defined.) 

Other  actions  are  envisioned  that  will 
also  improve  transition  and  help  strategic 
planning.  One  is  that  committee  assign¬ 
ments  will  be  posted  on  the  MORS  Web 
page  in  July.  A  second  is  that  committee 
POA&Ms  will  be  posted  in  August  and 
reviewed  by  the  Executive  Council  at  their 
September  meeting.  This  will  also  help 
our  future  volunteers  identify  areas  where 
they  can  help.  The  complete  set  of 
POA&Ms  for  the  Executive  Council  and 
their  committees  should  help  focus  our 
MORS  strategic  vision.  A  second  action  is 
a  MORS  President’s  Open  Forum  will  be 
held  in  the  Washington  area  —  a  lunch  or 
dinner  (Dutch  treat)  at  which  the  President 
(and  possibly  candidates  for  president  and 
president-elect  —  to  be  determined)  will 
answer  questions,  share  visions,  accept  rec¬ 
ommendations,  and  hold  discussions  with 
anyone  interested  in  MORS.  But,  as  we 
continue  to  make  recommendations  and 


take  actions  to  change  the  MORS  strategic 
plan,  we  must  remember  the  need  to  pro¬ 
tect  the  MORS  “niche”  —  the  capability  to 
hold  classified  meetings  in  areas  of  mili¬ 
tary  operations  research. 

One  of  the  past  actions  the  BOD  has 
taken  to  ensure  MORS  remains  focused  is 
to  elect  members  that  have  “made  a  differ¬ 
ence  to  MORS”  as  Fellows  of  the  Society. 
This  year  the  BOD  chose  three  new  Fel¬ 
lows:  Vern  Bettencourt,  FS,  Chris  Fos- 
sett,  FS,  and  Jackie  Henningsen,  FS  — 
all  of  whom  so  richly  deserve  the  recogni¬ 
tion  based  on  their  prior  actions  and  dedi¬ 
cation  to  MORS. 

The  challenge  to  keep  MORS  strategi¬ 
cally  focused  has  been  discussed.  Now  it 
is  time  for  members  and  volunteers  to  plan 
ahead.  In  the  near  term,  plan  to  attend  the 
next  MORSS  to  be  held  at  the  Naval  Post¬ 
graduate  School,  Monterey  California,  23- 
25  June  1998  and  participate  in  “Preparing 
for  Military  Operations  Research  in  the 
21st  Century.”  In  the  far  term,  plan  to  set 
goals  and  take  actions  to  keep  MORS  and 
militaiy  operations  research  relevant.  O 


VEEPS  PEEP 
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relevant  information  to  the  membership. 

Akin  to  the  “pull”  of  information  from 
the  web  has  been  the  use  of  e-mail  and 
mailing  lists  to  “push”  caplets  of  informa¬ 
tion  across  our  membership.  This,  too,  is  a 
capability  that  has  exploded  in  the  recent 
past.  Information  technology  is  no  doubt 
having  an  impact,  in  helping  us  become 
much  more  efficient  and  knowledgeable,  in 
helping  the  Society  to  accelerate  the 
exchange  of  our  ‘lifeblood’. 

Vision  —  the  Shared  Identity 

The  final  key  area  critical  to  the  preser¬ 
vation  and  growth  of  our  Society  is  the 
vision.  The  vision  is  “the  invisible  field” 
which  enables  our  organization  to  have  a 
shared  identity,  to  ensure  we  are  on  the 
same  playing  field  as  we  move  ahead.  Key 
to  this  has  been  the  Re-Engineering  of  our 
Composite/Working  Groups  and  our  very 
active  strategic  planning  process.  The 
recent  decision  of  the  Board  of  Directors  to 
elect  a  President  Elect,  who  will  lead  the 
strategic  planning  process  the  year  preced¬ 


ing  his  or  her  term  as  President,  will  further 
strengthen  our  Society  in  this  critical,  life 
sustaining  area. 

In  summary,  the  MORS  vital  signs  are 
healthy.  We  look  forward  to  turning  the 


millennium  with  strong  Sponsor  support 
and  a  renewed  enthusiasm  from  within  our 
membership.  We  are  preparing  ourselves 
for  the  21st  Century!  O 
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Figure  3:  ProModel  Display  (Note:  traffic  pattern  is  drawn  to  scale 
and  overlays  a  satellite  photo  of  the  airfield) 


QUICK  RESPONSE 

(continued  from  p.  11) 

Thanks  to  the  vision  of  Laughlin’s  lead¬ 
ers,  there  was  never  an  issue  of  visibility, 
or  support  of  the  project.  For  example, 
immediately  upon  the  team’s  arrival  at 
Laughlin,  the  Group  Commander  con¬ 
vened  a  discussion  session  including  every 
Squadron  Commander,  as  well  as  the 
Maintenance  Chief  (civilian  contractor)  to 
fully  explain  their  concerns  and  answer 
questions.  Many  factors  affected  pilot 
training  capacity  including  classroom 
space,  housing,  and  numbers  of  instructor 
pilots.  Increased  funding  could  solve  most 
of  these  problems,  but  the  fundamental 
limitations  of  runway  capacity  and  airspace 
limitations  could  not  be  solved  quickly 
with  increased  funding.  Environmental 
studies  for  changes  in  airspace  usage  alone 
could  take  years;  therefore,  existing  airfield 
operations  would  be  the  focal  point  for  the 
project. 

Clearly  runway  and  airspace  resources 
would  dominate  the  analysis.  Laughlin 
AFB  has  three  runways.  The  T-37s  use 
one  runway  and  the  T38s  and  T-ls  share 
the  other  two.  All  students  complete  T-37 
training  and  then  move  on  to  the  T-38  (for 
fighter/bomber  students)  or  the  T-1  (for 
tanker/transport  students). 

Initial  problem  framing  quickly  nar¬ 
rowed  to  T-37s  for  several  reasons.  First, 
the  T-37  squadron  generated  the  most  sor¬ 
ties  in  the  Wing.  The  T-38  squadron 
trained  only  a  fraction  (20-40%)  of  the 
students,  and  the  T-1  squadron  did  much  of 
its  training  off-station.  Second,  modeling 
the  T-38s  and  T-ls  could  not  be  done  inde¬ 
pendently  since  they  share  runways,  often 
simultaneously,  and  have  distinctly  differ¬ 
ent  operating  characteristics  (e.g.  operating 
airspeeds  and  pattern  procedures).  Due  to 
the  interdependence  between  the  T-38s  and 
T-ls,  a  quick  analysis  would  be  impossi¬ 
ble.  Since  time  was  of  the  essence,  it  was 
decided  that  a  realistic  T37  model  would 
be  far  more  valuable  than  an  incomplete, 
complex  model  of  all  aircraft  interactions. 
In  addition,  with  a  single  runway  and  high 
number  of  local  sorties  the  team  (and 
Laughlin  leadership)  intuitively  believed 
that  the  T37  operation  would  be  a  con¬ 
straining  factor  in  the  operating  environ¬ 
ment.  Therefore,  the  analysis  focused  on 
T-37  operations.  If  we  could  verify  that 
the  T-37  would  be  a  bottleneck  in  the  quest 
for  the  planned  1 100  pilot  production  level, 
we  would  at  least  prove  that  significant 


changes  would  be  needed  to  meet  overall 
production  goals. 

BuUding  the  Model 

We  began  building  the  model  by 
observing  the  airfield  operations.  Prelimi¬ 
nary  data  gathering,  such  as  taxi  times, 
takeoff  delays,  and  pattern  times  began. 
The  team  then  went  back  to  the  Academy 
to  complete  the  work.  Laughlin  agreed  to 
provide  further  data  as  requested.  Working 
a  thousand  miles  from  Del  Rio,  TX  was 
not  ideal,  but  little  about  this  project  was. 

We  used  Pro  Model  software  for  this 
project  due  to  its  ease  of  use  and  familiari¬ 
ty.  Every  operations  research  student  at 
the  Academy  uses  this  software  in  an  intro¬ 
ductory  simulation  course.  A  key  member 
of  our  team.  Second  Lieutenant  Brian  Riz- 
zoli  was  a  recent  OR  graduate  of  the  Acad¬ 
emy  and  had  experience  with  the  software 
package.  Lt  Rizzoli  accomplished  all  of 
the  programming  for  this  project. 

The  team  started  by  looking  at  the  air¬ 
field  and  its  surrounding  airspace.  Because 
Laughlin  AFB  is  in  rural  Texas,  the  base 
has  a  rare  luxury  in  today’s  environment: 
plentiful  airspace.  This  then  focused  our 
attention  on  the  traffic  pattern  operations. 
Although,  we  could  not  eliminate  the  train¬ 
ing  areas  completely  from  the  model,  we 
decided  to  model  the  23  training  areas  sim¬ 


ply  as  placeholders.  Another  key  issue  was 
whether  to  model  for  weather  and  mainte¬ 
nance  attrition.  Since  Laughlin  had  years 
of  historical  attrition  data,  we  decided  to 
model  for  perfect  conditions.  Once  we  had 
a  “perfect  condition”  capacity,  Laughlin 
could  add  in  the  attrition  factors  to  get  real¬ 
istic  annual  sortie  production. 

The  T-37s  actually  use  two  runways  for 
their  operations,  one  at  Laughlin  AFB  and 
an  auxiliary  field  about  ten  minutes  flying 
time  from  Laughlin.  The  pilots  refer  to  the 
auxiliary  field  as  “Wizard,”  and  we  used 
this  designation  in  the  model  also  (see  Fig¬ 
ure  3).  The  operations  of  the  two  runways 
are  almost  identical  and  once  we  had  a 
model  built  for  one  runway,  it  would  take 
very  few  modifications  to  build  a  model  of 
the  second  mnway’s  operations. 

In  order  to  model  the  T-37  operations 
we  had  to  understand  the  types  of  profiles 
that  were  flown  and  the  number  of  sorties. 
There  were  four  basic  types  of  sorties 
called  for  in  the  UPT  syllabus:  contact 
(basic  flying),  instrument,  formation,  and 
navigation.  Each  sortie  has  its  own  profile 
with  slight  variations.  Once  we  analyzed 
the  data  provided  by  the  T-37  squadron  we 
came  up  with  nine  sortie  profiles.  Laugh¬ 
lin  also  provided  us  with  the  percentage 
flown  of  each  type  of  sortie.  From  this 

(See  QUICK  RESPONSE,  jt?.  32) 
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Assumptions 

12.0  hours  daylight 
1 1 .2  Hour  Launch  Window 
Minimum  Sortie  Length  - 1.05  Hours 
Variance  -  8% 


Results  (Average  of  5  Simulation  Runs) 


Step  Rate  (planned  takeoff  interval 

2.5  min 

2.8  min 

Total  Sorties  Flown 

270 

290 

Average  Sortie  Duration 

1.28  hours 

1 .33  hours 

Breakouts  (conflict  resolution) 

73 

95 

Max  Aircraft  Airborne 

32 

39 

Max  Aircraft  in  Pattern 

10 

13 

Avg  Number  of  Landings 

6.07/sortie 

3.08/sortie 

Max  Wait  for  Takeoff 

1 1 .0  minutes 

32.1  minutes 

Average  Wait  for  Takeoff 

5.2  minutes 

16.3  minutes 

Figure  4:  Simulation  Run  Statistics 
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data  we  could  build  the  typical  sortie  pro¬ 
file.  For  example,  one  sortie  might  do  pat¬ 
tern  work  first  and  then  go  to  an  area  while 
another  sortie  might  reverse  the  order. 

Once  we  had  the  sortie  profiles,  we 
could  concentrate  on  the  actual  operations 
of  the  aircraft  in  the  air  and  on  the  ground. 
The  primary  concern  now  was  conflict  reso¬ 
lution.  We  knew  the  general  flight  paths  of 
the  aircraft,  but  we  had  to  have  solutions 
when  two  aircraft  tried  to  occupy  the  same 
space  at  the  same  time.  Even  more  than 
this,  we  had  to  apply  these  conflict  resolu¬ 
tion  procedures  when  two  aircraft  got  too 
close  to  each  other.  To  do  this  we  built  a 
3000  ft.  buffer  around  each  aircraft.  When 
the  buffer  was  violated,  the  aircraft  with  the 
lower  priority  according  to  the  “rules  of  the 
road”  had  to  breakout  from  the  pattern. 
These  breakout  procedures  came  straight 
from  the  pilot’s  operating  procedures  and 
were  straight-forward  to  implement. 

We  added  in  the  ground  operations  for 
queuing  and  taxi  procedures  and  from  here 
we  had  a  workable  model  (see  Figure  3). 
This  initial  design  process  for  the  model 
took  about  a  month  for  us  to  design.  Again 
remember,  this  is  a  part  time  project  for  the 
design  team. 

Problems 

One  of  the  first  problems  we  encoun¬ 
tered,  after  the  shell  of  the  model  was  built, 
was  deciding  on  an  appropriate  variance  for 
airspeed  control.  Initially  the  aircraft 
moved  around  the  model  at  a  discrete  air¬ 
speed  determined  by  the  operating  regula¬ 
tions.  However,  we  knew  that  pilots  do  not 
maintain  a  given  airspeed  exactly,  especial¬ 
ly  in  a  training  environment.  We  decided  to 
use  a  normal  distribution  for  airspeed,  but 
we  needed  a  variance.  Our  first  attempt  at 
determining  a  variance,  was  the  “old  fash¬ 
ioned”  way.  We  had  an  observer  go  out  to 
the  runway  with  a  stopwatch  and  time  vari¬ 
ous  pattern  operations.  There  were  three 
problems  with  this  approach.  First,  deter¬ 
mining  an  exact  point  in  space  on  which  to 
start  and  stop  timing  was  next  to  impossi¬ 
ble.  Second,  the  pattern  was  not  operating 
at  anywhere  near  capacity  at  the  time. 
When  the  pattern  is  empty,  the  instructor 
pilots  will  let  students  make  more  mistakes 
and  vary  from  their  assigned  airspeed  more 
than  when  the  pattern  is  full.  Third,  when 
the  pattern  is  operating  near  capacity  pilots 


will  make  adjustments  to  keep  their  spacing 
when  at  all  possible.  When  we  reviewed 
the  observation  data  we  got  a  very  large 
variance.  It  is  common  to  have  these  prob¬ 
lems  when  observing  a  system  that  is  lightly 
loaded  and  then  trying  to  extrapolate  the 
data  to  a  system  under  maximum  load. 

We  knew  that  our  data  was  suspect  and 
decided  to  tighten  the  variance  based  on 
trial  runs  that  seemed  realistic  to  both 
Laughlin  pilots  and  the  team.  We  also 
designed  the  model  such  that  the  variance 
for  airspeed  and  some  other  parameters  was 
an  easily  changeable  variable  in  the  model. 
This  model  design  allowed  us  to  test  the 
sensitivity  of  the  model  to  the  variance  and 
alter  the  model  if  we  were  able  to  get  better 
variance  data  in  the  future.  This  was  not  an 
ideal  methodology,  but  given  the  time  con¬ 
straints  and  the  lack  of  a  near  capacity  oper¬ 
ation  to  observe,  we  made  the  compromise. 
The  eventual  variance  we  decided  upon  was 
8%. 

A  second  problem  was  deciding  what 
was  an  acceptable  quality  level  for  each  sor¬ 
tie.  For  example,  we  could  increase  the 
launch  rate  so  that  the  average  wait  time  for 
takeoff  was  an  hour.  Given  the  fuel  and 
training  restrictions  this  meant  that  the  sor¬ 
tie  might  last  only  20  minutes  instead  of  the 
planned  1.2  hours.  The  final  solution  to  this 
problem  was  to  make  many  of  the  quality 
indicators  outputs  for  the  simulation.  This 
way  the  Laughlin  leadership  could  judge  for 


themselves  whether  a  given  launch  rate 
was  feasible. 

The  Finished  Product? 

By  mid-March  we  presented  the  T-37 
model  to  Laughlin.  This  was  about  three 
months  after  we  began  this  part-time  pro¬ 
ject.  To  date  Laughlin  has  run  over  300 
simulations  since  receiving  the  model.  We 
have  not  decided  whether  to  pursue  the 
other  half  of  the  model  (i.e.  T-38/T-1  oper¬ 
ations)  since  the  model  already  shows  sig¬ 
nificant  capacity  shortfalls  in  just  the  T-37 
operations.  The  typical  launch  rate  we 
achieved  was  in  the  range  of  a  sortie  every 
2.8  minutes  (see  Figure  4  for  typical  simu¬ 
lation  run  statistics).  From  the  simulation 
run  statistics  you  can  see  that  although  the 
number  of  sorties  flown  increases  with 
increased  step  rate,  the  quality  statistics  go 
down  significantly.  Specifically,  takeoff- 
wait  increases  and  number  of  landings  per 
sortie  decreases  (a  maximum  number  of 
landings  is  essential  for  quality  training  in 
the  UPT  environment).  This  is  significant¬ 
ly  less  than  the  previous  estimate  of  a  sortie 
every  2.5  minutes. 

Conclusion 

Due  to  this  study  and  other  indicators, 
AETC  is  currently  looking  at  other  ways  to 
ease  the  capacity  problem  at  its  three  pri¬ 
mary  UPT  bases.  Some  of  these  options 
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include:  going  to  single  aircraft  operations 
for  each  base  (i.e.  all  T-37s  at  one  base, 
etc.);  moving  T-1  operations  to  a  new  base; 
and  possible  using  Randolph  AFB  for 
some  pilot  training  operations.  No  deci¬ 
sion  on  an  acceptable  resolution  for  this 
issue  has  been  made  to  our  knowledge. 
Providing  a  solution  to  the  problem  was 
not  our  goal.  Our  goal  was  to  give  Laugh- 
lin  a  tool  that  they  could  use  to  estimate 
their  true  training  capacity  (and,  by  the 
way,  do  it  quickly).  In  this  goal  we  suc¬ 
ceeded.  Here  are  a  few  of  the  key  points 
that  made  our  quick-response  project  pos¬ 
sible: 

1.  We  were  able  to  narrow  the  scope  of 
the  project  quickly  by  establishing 
likely  bottleneck  points. 

2.  We  had  an  operational  expert  (former 
T-38  Instructor)  on  our  design  team. 

3.  An  exact  solution  was  not  needed  (i.e. 
the  total  capacity  for  the  base).  The 
user  only  wanted  to  validate  the 
hypothesis  that  the  training  capacity 
would  fall  short  of  projections. 

Obviously,  every  project  will  not  be 
suitable  for  this  quick  response  approach. 
However,  under  certain  circumstances,  a 
simulation  may  be  an  appropriate  response 
for  a  problem  that  needs  a  quick  answer. 
With  today’s  modem  software  tools,  simu¬ 
lation  is  moving  into  the  mainstream  for 
many  projects  that  were  not  feasible  for 
this  approach  in  the  past. 
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cult  to  describe.  Allied  and  coalition  opera¬ 
tions  predominate,  yet  none  are  certain  apri- 
ori.  We  as  a  Defense  culture  simply  face  a 
bewildering  mix  of  complexity.  The  central 
issues  are  simply  not  as  clear  as  they  used  to 
seem. 

The  first  and  most  important  job  of  the 
analyst  is  to  get  the  questions  right.  My 
Dad  once  told  me  that  if  the  answers  I  was 
getting  weren’t  solving  my  problems,  then 
maybe  I  was  asking  the  wrong  questions.  I 
see  a  lot  of  answers  being  generated  by 
studies  today,  but  I  also  see  us  asking  the 
same  questions  over  and  over  in  repetitive 
studies  like  DAWMS  and  MRS,  BASE 
FORCE,  BUR,  QDR,  WMD,  CBMR.  That 
leads  me  to  think  that  maybe  we  haven’t  got 
the  questions  right  yet. 

It’s  no  surprise.  The  issues  we  face 
today  are  so  new  and  complex  that  I  suspect 
that  frequently  the  decision  makers  aren’t 
that  sure  what  it  is  they  need  to  ask.  The 
analyst  needs  to  be  diplomatically  aware 
that  initial  analytic  tasking  can  be  just  a  des¬ 
perate  stab  at  a  general  lack  of  insight,  and 
as  such  can  be  somewhat  removed  from  the 
underlying  central  issues,  the  “real  ques¬ 
tion.” 

Always,  “Look  before  you  leap.”  If  the 
analyst  jumps  to  answer  the  questions  as 
they  are  first  asked  today,  he  or  she  mns  a 
real  risk  of  producing  irrelevant  analysis.  I 
consider  it  an  analyst’s  obligation  to  ques¬ 
tion  the  question. 

How  can  an  analyst  discover  the  right 
questions?  On  the  Joint  Staff,  we  are  using 
seminar  gaming  with  the  real  decision  mak¬ 
ers  as  a  “front  end”  to  the  quantitative 
analysis.  The  evolving  experience  is  that 
any  analysis  can  benefit  by  engaging  the 
decision  makers  and  stakeholders  early  in 
the  design  phase  of  the  analysis. 

There  is  an  important  point  to  be  made 
here.  “Knowing  your  customer”  means 
more  than  knowing  “who”  is  chartering  the 
study.  It  means  helping  the  stakeholders 
and  decision  makers  discover  what  it  is  they 
really  need  to  know.  And  it  means  doing 
that  before  you  design  the  study  methodolo¬ 
gy- 

Seminar  gaming  is  proving  to  be  an 
excellent  method  for  discovering  the  real 
questions  that  underlie  the  issues.  With 
increasing  frequency,  we  are  using  seminar 
gaming  as  a  front  end  to  our  quantitative 
analyses.  We  get  the  stakeholders  and  deci¬ 


sion  makers  to  the  table;  immerse  them  in  a 
scenario  containing  their  initial  issues  and 
have  them  work  through  that  scenario  to  a 
better  understanding  of  what  they  are  really 
interested  in.  In  this  way,  we  and  they  dis¬ 
cover  what  it  is  about  the  general  issue  that 
is  really  important,  what  the  range  of  viable 
solution  actions  might  be,  and  how  the 
important  relationships  and  measures  might 
best  be  presented  to  them.  In  this  way,  sem¬ 
inar  gaming  at  the  front  end  of  the  analysis 
helps  us  cover  the  first  three  steps  of  the 
analysis  recipe;  “What’s  the  Question,” 
“What’s  the  Real  Question,”  and  “What  do 
the  Final  Slides  look  like.” 

The  Third  Step  in  the  analytic 
process,  ‘What  do  the  Final  Slides 
Look  like?”  —  may  need  some  clari¬ 
fication. 

Understanding  what  the  final  slides  must 
look  like  does  not  mean  to  suggest  that  the 
analyst  has  the  “answer”  before  he  does  the 
analysis.  It  means  that  he  must  know  which 
key  relationships  and  perspectives  need  to 
be  presented  to  illuminate  the  “Real  Ques¬ 
tions.”  With  this  view  of  the  final  slides,  the 
analyst  can  determine  what  data  enables 
those  relationships  and  perspectives  to  be 
developed.  Once  an  analyst  knows  what 
the  real  questions  are  and  how  to  present 
key  relationships,  it  is  not  uncommon  to 
discover  that  most  of  the  data  needed 
already  exist. 

The  Fourth  Step  in  the  analysis 
recipe,  “What  do  I  already  know?” 

—  is  the  Literature  Search  phase  of 
the  analysis. 

The  literature  search  may  be  the  most 
misused  of  the  analysis  recipe  steps, 
because  if  it  is  done  at  all,  it  is  often  done  as 
the  second  step  in  an  analysis  process  right 
after,  “What’s  the  Question?”  And,  unfor¬ 
tunately  it  may  be  done  not  to  discover  what 
is  already  known  about  the  key  relation¬ 
ships,  but  to  see  what  other  analysts  may 
have  done  to  answer  the  initial  question. 
There  is  a  false  safety  in  those  precedents. 
Yesterday’s  methods  and  answers  may  be 
of  little  help  in  illuminating  today’s  central 
issues.  When  the  literature  search  is  done 
as  the  fourth  step  in  the  analysis  process,  the 
analyst  knows  what  to  look  for,  why  it’s 
needed,  and  how  to  use  it. 

ANALYSIS,/?.  34) 
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The  Fifth  Step,  ‘TIow  do  I  get  the 
remaining  information  I  need?” 
comes  into  play  when  data  to  fill  in 
the  final  slides  aren’t  avaUable  and 
must  be  generated. 

This  step  may  be  where  an  analysis  is 
most  likely  to  become  irrelevant. 

More  than  any  other  time  I  can  recall, 
the  Defense  Analysis  community  seems  to 
have  become  excessively  ‘Model  Centric” 
in  its  behavior.  Too  often  the  first  thought 
from  the  analyst  when  faced  with  a  tasking 
is,  “What  model  do  I  use?”  To  make  mat¬ 
ters  more  dangerous,  there  is  an  extensive 
culture  of  (Modeling  &  Simulation)  M&S 
advocates,  committee  bureaucracies,  and 
W&A  safeguards  encouraging  the  analyst 
toward  Model  Centric  thinking. 

Analysts  were  once  valued  primarily  for 
the  way  they  thought.  The  current  emphasis 
toward  model  centric  analysis  almost 
encourages  the  analyst  not  to  think.  The 
perception  is  growing  that  with  enough 
improvement,  models  will  become,  “bozo 
proof’  —  meaning  that  one  won’t  have  to 
think  much  in  order  to  use  them  and  pro¬ 
duce  what  can  pass  for  sound  analytic 
insight.  After  all,  if  the  data  are  blessed, 
and  the  Model  is  W&A’d,  then  anyone  can 
turn  the  crank  and  produce  insights.  Maybe 
most  dangerous  of  all  is  that  senior  decision 
makers  are  beginning  to  think  that  using  a 
tmsted  model  equates  to  generating  a  quali¬ 
ty  analysis.  Such  thoughts  can  be  danger¬ 
ous.  A  better  model  improves  the  analysis 
only  if  the  analysis  process  is  correct  for  the 
issue  being  examined. 

How  can  analysts  overcome  model  cen¬ 
tric  thinking  and  make  their  analyses  more 
relevant?  For  one,  they  can  follow  an  analy¬ 
sis  recipe  such  as  enumerated  here.  But 
there  is  one  more  basic  principle  that  can 
help.  That  principle  is  “Collaboration.” 

Collaboration  is  more  than  coordination. 
Collaboration  acknowledges  that  good 
analysis  has  at  least  two  important  func¬ 
tions.  One,  of  course,  is  to  help  the  decision 
maker  better  understand  the  issues  and  to 
appreciate  the  consequences  of  potential 
decisions.  The  other  important  function, 
which  analyses  many  times  fails  to  do,  is  to 
help  prepare  the  stakeholders  to  implement 
the  decisions  when  they  are  made.  Collabo¬ 
ration  supports  both  of  these  functions. 

Collaboration  engages  stakeholders  in 


the  design  and  the  execution  of  the  analysis. 
It  recognizes  that  there  are  many  potential 
views  and  interests  in  any  complex  issue 
and  that  the  best  experts  and  tools  in  these 
interests  are  usually  in  the  hands  of  the 
stakeholders.  As  an  example,  it  would  be 
better  to  engage  the  lift  community  and 
their  models  directly  in  the  transportation 
aspects  of  a  global  warfare  capability  analy¬ 
sis  than  to  run  one’s  own  model  based 
transportation  analysis.  Even  if  the  analysts 
use  the  same  tools,  if  the  transportation 
expertise  wasn’t  collaborating  in  the  design 
and  execution  of  the  analysis,  the  analyst 
could  readily  expect  that  when  the  study 
was  done  and  offered  for  coordination, 
those  he  ignored  would  arrive  at  his  work 
with  drawn  knives  and  debate  and/or 
counter  the  findings  rather  than  help  the 
decision  maker  implement  any  decision 
coming  from  the  work.  Community  owner¬ 
ship  is  an  important  element  in  most  rele¬ 
vant  analyses.  Collaboration  can  provide 
that  sense  of  ownership. 

In  short,  without  collaboration,  all  a 
major  study  is  likely  to  give  the  decision 
maker  is  a  bigger  debate;  and  another  irrele¬ 
vant  study.  Unfortunately  that  resulting 
debate  frequently  takes  the  form  of  an 
attack  on  the  analyst’s  model.  That  leads  the 
decision  maker,  in  frustration,  to  give  direc¬ 
tion  to  “fix”  the  models,  instead  of  to  a  deci¬ 
sion  on  the  real  questions.  This,  then,  fur¬ 
ther  fosters  the  notion  that  a  better  model 
will  lead  to  a  better  analysis;  more  model 
centric  thinking.  The  model  may  have  been 
fine;  it  was  the  analysis  process  that  needed 
fixing. 

Even  if  the  model  gets  “fixed,”  and  that 
can  be  expensive  (try  the  better  part  of  $100 
million  for  JWARS),  if  the  next  analysis 
uses  the  model  in  isolation  from  the  stake¬ 
holders,  the  resulting  analysis  could  easily 
just  generate  another  model  centric  debate. 
Sadly,  the  more  one  spends  to  “fix  the 
model”  the  more  one  is  obligated  to  use  it 
on  the  next  analytic  question  —  still  more 
model  centric  encouragement.  It’s  insidious. 
The  analyst  must  not  become  wed  to  his 
tools.  He  has  to  be  prepared  to  say,  “no”  to 
even  the  most  vigorous  Model  Centric  pres¬ 
sures  if  the  models  don’t  fit  the  analysis 
design  at  hand. 

I  agree  with  those  who  argue  that  better 
models  are  desirable.  But,  I  do  not  believe 
that  better  models,  by  themselves,  will  give 
us  better  analyses.  The  path  to  relevant 
analyses  lies  in  better  analysis  process  and 
collaboration,  not  just  with  better  models. 


These  thoughts  on  collaboration  and  use 
of  an  analysis  recipe  may  have  significant 
implications  for  the  new  JWARS  model  and 
its  ability  to  improve  joint  analysis.  My 
sense  is  that  the  “stronger”  a  model  is  pur¬ 
ported  to  be  and  the  more  universally  it  is 
endorsed,  the  greater  the  temptation  is  to 
use  it  in  isolation;  “Why  do  we  need  to 
involve  them?  The  model  already  represents 
how  they  do  their  part.”  Doing  analysis  in 
noncollaborative  ways  sets  the  stage  for 
studies  becoming  irrelevant,  regardless  of 
how  robust  the  model  that  was  used. 

So,  what’s  the  bottom  line  here?  It  is 
this.  Relevant  studies  tend  to  address  two 
key  areas;  increasing  capability  and 
decreasing  costs.  A  study  can  be  made 
more  relevant  by  making  it  “actionable,” 
that  is  that  it  encourages  specific  actions  or 
decisions.  Relevant  studies  tend  to  follow  a 
process  of  analysis,  a  recipe,  that  starts  with 
identifying  the  “real”  questions  and  know¬ 
ing  what  the  final  slides  have  got  to  look 
like  before  one  starts  looking  for  data  or 
generating  new  data.  Finally,  relevant  stud¬ 
ies  tend  to  be  inclusive,  collaborative  activi¬ 
ties.  Collaboration  helps  get  better  expertise 
into  the  study;  helps  ensure  that  the  study  is 
comprehensive;  and  it  helps  prepare  the 
stakeholders  to  support  implementation  of 
any  decision  resulting  from  the  study.  Last¬ 
ly,  analysts  who  produce  relevant  studies 
have  avoided  or  broken  out  of  the  “model 
centric”  mentality.  They  have  practiced  the 
analytic  art.  Remember,  it  isn’t  because  we 
have  the  world’s  greatest  models  (which  we 
do);  “They  keep  us  analysts  around, 
because  they  like  the  way  we  think!”.  That 
last  quote  came  from  Dave  Robinson,  too. 
And  you  know  what?  He  was  right.  O 


The  theme  for  the  Education 
Colloquium  this  year  is ... 

"Sharpening  the  Saw  ~ 
Maintaining  OR  Professional 
Readiness  &  Relevance." 

It  will  be  held 
21-22  April  1998 

at  the  ANSER  offices 
located  in  Crystal  City  Virginia 

If  you  would  like  more  information 
on  this  event  please  call  COL  Ken¬ 
neth  Konwin  at  703-998-0660. 
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THE  LAST  WORD 


Farewell  to  Jack  Walker 

Dick  Wiles 


When  I  came  to  MORS  almost 
fourteen  years  ago,  Jack  Walk¬ 
er  was  one  of  the  first  MOR- 
Sians  (his  term)  I  met.  Jack  was  the  Editor 
of  PHALANX.  I  was  the  new  Executive 
Director.  Jack  showed  me  how  to  put  the 
PHALANX  together,  literally.  Jack  was  not 
only  the  Editor  but  also  the  Composer.  The 
PC  and  MAC  were  just  coming  on  the 
scene.  Composition  was  a  mechanical,  not 
electronic  process.  Natalie  typed  the  text 
into  columns  which  Jack  pasted  (actually 
waxed)  to  pages  which  went  to  the  printer. 

After  a  couple  of  issues.  Jack  submitted 
his  first  resignation  as  editor.  One  of  my 
better  decisions  that  year  was  to  reject  his 
resignation.  The  following  year  Jack  sub¬ 
mitted  his  resignation  to  newly  elected  Pres¬ 
ident  Wayne  Hughes.  Wayne  prudently 
rejected  the  resignation  (see  “A  Tribute  to 
Jack  from  Wayne  Hughes,”  pg  16).  It 
became  clear  to  me  that  this  resignation 


business  was  going  to  be  an  annual  affair. 
I  made  it  my  business  when  orienting 
newly  elected  presidents  to  tell  them  to 
expect  a  resignation  from  Jack  but  to  be 
sure  to  reject  it. 

Later,  when  Jack  and  Tony  started  win¬ 


tering  in  Florida,  it  became  clear  that  he 
did  really  want  to  retire.  He  was  told  that 
he  could  retire  as  soon  as  we  found  a 
replacement.  He  agreed  to  serve  on  the 
search  committee  that,  about  a  year  later, 
found  his  successor,  Dee  Ritchie. 

Jack  was  editor  for  over  twelve  years 
(the  longest  tenure  of  all  the  editors).  His 
first  issue  in  August  1979  (Vol  12  No  3) 
was  just  twelve  pages.  His  last  in  Decem¬ 
ber  1991  (Vol  24  No  4)  was  thirty-six 
pages.  When  he  retired  (as  Editor),  MORS 
presented  him  a  MORS  chair,  two  bound 
volumes  of  his  PHALANXs  and  made  him 
Editor  Emeritus.  But  Jack  did  not  retire 
fully.  He  continued  as  a  Department  Edi¬ 
tor  and  when  MORS  found  itself  “between 
editors”  Jack  pitched  in  as  Acting  Editor. 

While  Dee  Ritchie  and  Julian  Palmore 
have  both  put  their  own  unique  marks  on 
PHALANX,  it  was  Jack  Walker  who  made 
it  what  it  is.  Jack  will  be  missed.  O 
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